About parameter --rna

[gremory-x]

Hi,

Thanks for the useful tool.
I am currently uncertain about the optimal type of transcriptome files to provide for my study. I am considering two approaches: directly using StringTie/Trinity for transcriptome assembly and extracting the corresponding FASTA files, or performing additional steps using TransDecoder to predict open reading frames (ORFs) and then filtering them against databases like SwissProt. The examples I reviewed include filenames with "curated," which has piqued my curiosity about the appropriate files I should provide.

Gaorui

[wtroy2]

Id go with option 1 over option 2. But you can do either.

When saying curated what was meant was, if not using a transcript of the species of interest, we generally combined, just by catting, up to 3 related species transcript files together and input that for rna.

FLAG can also run trinity internally but at this time it only takes fasta for input to trinity if that is desired so it may be better to run trinity outside of it if you already have trinity setup on your own

[gremory-x]

Thanks for your reply. I'll give it a try.

[CEPHAS-01]

Hi
Thanks for this tool.
I am posting this under this heading because it is related to it.
I am about to give it a try now and would like to use the --rna parameter with raw rnaseq fasta files. My reads are paired-end and it is suggested that the rna files should be supplied as a single file (I understand I have to concatenate multiple files from multiple experiments into one). Since trinity has parameters to take paired-end reads, how would this be handled internally by trinity. Do I still proceed to concatenate ALL files (multiple experiments) into a single raw rnaseq reads file?

Thank you in advance for your answer.

TLag

[wtroy2]

Hi TLag, The current open source version of FLAG only supports fasta file inputs for trinity to make transcripts. Due to this you currently have 3 options. 1. Convert your fastq reads to fasta format and input this into flag 2. Run trinity outside of flag to make a transcript and then input this transcript into flag 3. I can probably add support for fastq inputs with multiple files in the next week or two if you are OK with waiting a little Let me know which is preferable for you and I can try to help guide you through whichever. Will

…

On Sun, Feb 18, 2024, 5:31 PM Temitayo Olagunju ***@***.***> wrote: Hi Thanks for this tool. I am posting this under this heading because it is related to it. I am about to give it a try now and would like to use the --rna parameter with raw rnaseq fasta files. My reads are paired-end and it is suggested that the rna files should be supplied as a single file (I understand I have to concatenate multiple files from multiple experiments into one). Since trinity has parameters to take paired-end reads, how would this be handled internally by trinity. Do I still proceed to concatenate ALL files (multiple experiments) into a single raw rnaseq reads file? Thank you in advance for your answer. TLag — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHT22X6ROPFZEWXIGBORG33YUKFO5AVCNFSM6AAAAAA2VEJCMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNJRGQ4DGMRWGM> . You are receiving this because you commented.Message ID: ***@***.***>

[CEPHAS-01]

Hi Will,

Thanks so much for your prompt response.

First, I proceeded with concatenating all the fasta files. However, I was not successful with the run.
I will post the error message below:

Secondly, you mentioned "open source version"...is there a proprietary version that could be used? If yes, what are the extra features that this has but absent in this open source version. What does it take to have access to the other version?

Running with the singularity version:

nextflow run main.nf -w $4 --output $5 --genome $1
--masker skip
--lineage cetartiodactyla_odb10
--annotationalgo Helixer,Liftoff
--helixerModel vertebrate
--proteins $2
--rna $3
--transcriptIn false
--externalalgo input_transcript,input_proteins
--size normal
--proteinalgo miniprot
--speciesScientificName Ovis_aries
-profile docker

The error:
N E X T F L O W ~ version 23.10.1
Launching main.nf [nice_euler] DSL2 - revision: 0c734bb472
WARN: Access to undefined parameter fafile -- Initialise it to a default value eg. params.fafile = some_value
WARN: Access to undefined parameter gtffile -- Initialise it to a default value eg. params.gtffile = some_value
WARN: Access to undefined parameter blastdb -- Initialise it to a default value eg. params.blastdb = some_value
WARN: Access to undefined parameter rnaDB -- Initialise it to a default value eg. params.rnaDB = some_value

Test - N F P I P E L I N E

outdir : flagAnnotationOut
masker : skip
genome : genomic.fasta
proteins : Protein.fa
rna : all.fasta
reference_genome : null
reference_annotation : null
protein database : null
rna database : null
transcriptIn : false
Busco Lineage : cetartiodactyla_odb10
entapDB : entapDBs.tar.gz
Augustus Pretrained Species : human
Helixer Model : vertebrate
Helixer Models Available : verterbrate, invertebrate, land_plant, fungi
Genome Size : normal
Species Scientific Name : Ovis_aries

all annotation algos options : Helixer, Liftoff, denovo_augustus, related_species_augustus, augustus_pretrained, liftoff_trained_augustus, helixer_trained_augustus, transdecoder
chosen annotation algos : Helixer,Liftoff

all external algos options : input_transcript, input_proteins, transcript_from_database, proteins_from_database
chosen external algos : input_transcript,input_proteins

all protein algos : exonerate, genomethreader, prosplign, miniprot
chosen protein algos : miniprot

WARN: Access to undefined parameter runMode -- Initialise it to a default value eg. params.runMode = some_value
[- ] process > trnascan -
[- ] process > protein_removedupsfa -
[- ] process > miniprot -
[- ] process > rna_removedupsfa -
[- ] process > trinity -
[- ] process > pasa -
[- ] process > splign -
[- ] process > Helixer -
[- ] process > CombineAndFilter -
[- ] process > entap -

[- ] process > trnascan -
[- ] process > protein_removedupsfa [ 0%] 0 of 1
[- ] process > miniprot -
[- ] process > rna_removedupsfa [ 0%] 0 of 1
[- ] process > trinity -
[- ] process > pasa -
[- ] process > splign -
[- ] process > Helixer -
[- ] process > CombineAndFilter -
[- ] process > entap -
[- ] process > combinestructwfunct -

executor > Local (4)
[a1/bd4c27] process > trnascan (1) [100%] 1 of 1, failed: 1 ✔
[8e/16c785] process > protein_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > miniprot -
[dd/b1f296] process > rna_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > trinity -
[- ] process > pasa -
[- ] process > splign -
[7b/dd7b3f] process > Helixer (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > CombineAndFilter -
[- ] process > entap -
[- ] process > combinestructwfunct -
[8e/16c785] NOTE: Process protein_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[dd/b1f296] NOTE: Process rna_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[a1/bd4c27] NOTE: Process trnascan (1) terminated with an error exit status (127) -- Error is ignored
[7b/dd7b3f] NOTE: Process Helixer (1) terminated with an error exit status (127) -- Error is ignored

executor > Local (5)
[a1/bd4c27] process > trnascan (1) [100%] 1 of 1, failed: 1 ✔
[8e/16c785] process > protein_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > miniprot -
[dd/b1f296] process > rna_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > trinity -
[- ] process > pasa -
[- ] process > splign -
[7b/dd7b3f] process > Helixer (1) [100%] 1 of 1, failed: 1 ✔
[cd/94b85d] process > CombineAndFilter (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > entap -
[- ] process > combinestructwfunct -
[8e/16c785] NOTE: Process protein_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[dd/b1f296] NOTE: Process rna_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[a1/bd4c27] NOTE: Process trnascan (1) terminated with an error exit status (127) -- Error is ignored
[7b/dd7b3f] NOTE: Process Helixer (1) terminated with an error exit status (127) -- Error is ignored
[cd/94b85d] NOTE: Process CombineAndFilter (1) terminated with an error exit status (127) -- Error is ignored

executor > Local (5)
[a1/bd4c27] process > trnascan (1) [100%] 1 of 1, failed: 1 ✔
[8e/16c785] process > protein_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > miniprot -
[dd/b1f296] process > rna_removedupsfa (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > trinity -
[- ] process > pasa -
[- ] process > splign -
[7b/dd7b3f] process > Helixer (1) [100%] 1 of 1, failed: 1 ✔
[cd/94b85d] process > CombineAndFilter (1) [100%] 1 of 1, failed: 1 ✔
[- ] process > entap -
[- ] process > combinestructwfunct -
[8e/16c785] NOTE: Process protein_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[dd/b1f296] NOTE: Process rna_removedupsfa (1) terminated with an error exit status (127) -- Error is ignored
[a1/bd4c27] NOTE: Process trnascan (1) terminated with an error exit status (127) -- Error is ignored
[7b/dd7b3f] NOTE: Process Helixer (1) terminated with an error exit status (127) -- Error is ignored
[cd/94b85d] NOTE: Process CombineAndFilter (1) terminated with an error exit status (127) -- Error is ignored

I did not provide an Entap DB as I saw that it was not compulsory...or is it?

Thank you in advance for your help.

Warm regards,

TLag

[wtroy2]

Hey,
So a few things. First yes there is another version that we update and run more often available on the form bio platform. You can run it there with a point a click interface without installing anything. Databases are already pre configured in that one as well including transcript databases. For first time google cloud users you also get $300 free so it should cover most if not all of your run hopefully: https://console.cloud.google.com/marketplace/product/formbio-public/form-bio

Entap database is not required but just know for functional annotation and the combined structural and functional output at the end it is required.

On your run you are running it with docker not singularity. Youd need to change the profile from docker to singularity so your final run command would be:
nextflow run main.nf -w $4 --output $5 --genome $1
--masker skip
--lineage cetartiodactyla_odb10
--annotationalgo Helixer,Liftoff
--helixerModel vertebrate
--proteins $2
--rna $3
--transcriptIn false
--externalalgo input_transcript,input_proteins
--size normal
--proteinalgo miniprot
--speciesScientificName Ovis_aries
-profile singularity

I know a couple users are currently having trouble with singularity so im working to try to iron that out as well

[CEPHAS-01]

Hi @wtroy2 ,

Thanks for the prompt feedback.

Indeed I had used -profile singularity at some point and still got an error before trying out -profile docker.

Anyway, I tried again and got the following error in addition to a similar output to the one I posted earlier.

WARN: Unknown directive runOptions for process pasa
[d8/72e20a] NOTE: Process rna_removedupsfa (1) terminated with an error exit status (255) -- Error is ignored
[78/d76a78] NOTE: Process protein_removedupsfa (1) terminated with an error exit status (255) -- Error is ignored
[e3/f675a3] NOTE: Process Helixer (1) terminated with an error exit status (255) -- Error is ignored
[b9/dbd123] NOTE: Process trnascan (1) terminated with an error exit status (255) -- Error is ignored
[77/a56d8a] NOTE: Process CombineAndFilter (1) terminated with an error exit status (255) -- Error is ignored

Perhaps this is the current challenge with singularity you mentioned.

Thanks for your help.

TLag

[wtroy2]

Hey @CEPHAS-01 I just pushed a bunch of updates for singularity support and hopefully to make the process easier, note the run command has been updated as well for functional annotation as Eggnog is now the default functional annotation algorithm since the database is more up to date than the eggnog database used in EnTAP.

If you could pull the changes to the main branch and then rebuild the singularity images you may have better luck. Its setup now so that you should be able to build all singularity images by running:
"bash direct_pull_singularity_images_and_move_to_folders.sh"
also make sure you have the initial directory structure setup with:
"bash makeDirectories.sh"

The eggnog database is also required for functional annotation but it is easy to build and all you should need to do is run
"bash setup_eggnogDB.sh" and it should build for you as its just downloading public files and formatting them.

These setups are outlined in the updated README.md file, shown in the image attached:

Note that the run command was also slightly updated for the EggnogDB and functional annotation program. This updated command is also in the new README.md and in the image below although its cutoff some in the attached image:

Hopefully this allows you to run it now. Error 255 can be a bunch of things but I'm hopeful with the setup scripts and updates that it will work. If it doesn't let me know and I may need more detailed logs to try to debug since 255 can be anything from permissions to files not existing or other things.

[CEPHAS-01]

Hi William @wtroy2 ,

Thanks so much for providing the updates to the container. I will take a dive into this ASAP and give you feedback!

TLag

[CEPHAS-01]

Hi @wtroy2
I followed the setup as you have highlighted but still could not run it through.

One error that stands out (see the log below) is the one about .."Script base path does not exist or is not a directory: /flagAnnotation2/FLAG/bin"

The /bin directory does not exist actually but there is a script directory here /flagAnnotation2/FLAG/scripts/

FWIW, my singularity version is 1.1.6-1.el7

My command

nextflow run /flagAnnotation2/FLAG/main.nf -w /flagAnnotation2/FLAG --output /flagAnnotation2/flagAnnotationOut --genome genomic.fasta
--masker skip
--lineage cetartiodactyla_odb10
--annotationalgo Helixer,Liftoff
--helixerModel vertebrate
--proteins $2
--rna $3
--transcriptIn false
--externalalgo input_transcript,input_proteins
--size normal
--proteinalgo miniprot
--speciesScientificName Ovis_aries
--funcAnnotProgram eggnog
--eggnogDB eggnogDB.tar.gz
-profile singularity

The beginning of the error log:
Mar-31 19:45:47.032 [main] DEBUG nextflow.cli.Launcher - $> nextflow run /flagAnnotation2/FLAG/main.nf -w /flagAnnotation2/FLAG/scripts --output /flagAnnotation2/flagAnnotationOut --genome genomic.fasta --masker skip --lineage cetartiodactyla_odb10 --annotationalgo Helixer,Liftoff --helixerModel vertebrate --proteins hsaMmuBtaOarProtein.fasta --rna all.fasta --transcriptIn false --externalalgo input_transcript,input_proteins --size normal --proteinalgo miniprot --speciesScientificName Ovis_aries --funcAnnotProgram eggnog --eggnogDB eggnogDB.tar.gz -profile singularity
Mar-31 19:45:47.267 [main] INFO nextflow.cli.CmdRun - N E X T F L O W ~ version 23.10.1
Mar-31 19:45:47.290 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/.nextflow/plugins; core-plugins: nf-amazon@2.1.4,nf-azure@1.3.3,nf-cloudcache@0.3.0,nf-codecommit@0.1.5,nf-console@1.0.6,nf-ga4gh@1.1.0,nf-google@1.8.3,nf-tower@1.6.3,nf-wave@1.0.1
Mar-31 19:45:47.302 [main] INFO o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Mar-31 19:45:47.303 [main] INFO o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Mar-31 19:45:47.307 [main] INFO org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Mar-31 19:45:47.319 [main] INFO org.pf4j.AbstractPluginManager - No plugins
Mar-31 19:45:47.339 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /flagAnnotation2/FLAG/nextflow.config
Mar-31 19:45:47.342 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /flagAnnotation2/FLAG/nextflow.config
Mar-31 19:45:47.376 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: singularity
Mar-31 19:45:48.440 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [docker_small, singularity_small, singularity, docker]
Mar-31 19:45:48.483 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declararion
Mar-31 19:45:48.503 [main] INFO nextflow.cli.CmdRun - Launching /flagAnnotation2/FLAG/main.nf [astonishing_raman] DSL2 - revision: 6332396f7e
Mar-31 19:45:48.504 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Mar-31 19:45:48.504 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Mar-31 19:45:48.515 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /.nextflow/secrets/store.json
Mar-31 19:45:48.520 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@58b91d57] - activable => nextflow.secret.LocalSecretsProvider@58b91d57
Mar-31 19:45:48.580 [main] DEBUG nextflow.Session - Session UUID: a0f60be2-d124-44c1-85ad-d5dca0206b0c
Mar-31 19:45:48.580 [main] DEBUG nextflow.Session - Run name: astonishing_raman
Mar-31 19:45:48.581 [main] DEBUG nextflow.Session - Executor pool size: 40
Mar-31 19:45:48.588 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Mar-31 19:45:48.619 [main] DEBUG nextflow.cli.CmdRun -
Version: 23.10.1 build 5891
Created: 12-01-2024 22:01 UTC (14:01 PDT)
System: Linux 3.10.0-1160.90.1.el7.x86_64
Runtime: Groovy 3.0.19 on OpenJDK 64-Bit Server VM 21.0.2+13-LTS
Encoding: UTF-8 (UTF-8)
Process: 41928@n045
CPUs: 40 - Mem: 251.7 GB (235.8 GB) - Swap: 2 GB (1.2 GB) - Virtual threads ON
Mar-31 19:45:48.661 [main] DEBUG nextflow.Session - Work-dir: /flagAnnotation2/FLAG/scripts [node]
Mar-31 19:45:48.662 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /flagAnnotation2/FLAG/bin
Mar-31 19:45:48.673 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Mar-31 19:45:48.688 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Mar-31 19:45:48.750 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Mar-31 19:45:48.761 [main] DEBUG nextflow.util.CustomPoolFactory - Creating virtual thread pool
Mar-31 19:45:49.012 [main] DEBUG nextflow.Session - Session start
Mar-31 19:45:49.024 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow started -- trace file: /flagAnnotation2/pipeline_trace.txt
Mar-31 19:45:49.060 [main] DEBUG nextflow.Session - Using default localLib path: /flagAnnotation2/FLAG/lib

Thank you in advance.

[wtroy2]

@CEPHAS-01 it appears others are also having singularity issues but everyone's issue is different so Im trying to make it runnable from a single singularity container so there's less orchestration necessary. Will update in the next couple of days once done testing it

[wtroy2]

Hey @CEPHAS-01 just so I make sure I'm getting the testing right for you are you singularity or apptainer version 1.1.6-1.el7? Singularity version 1.1.6 is incredibly old.

[CEPHAS-01]

Hi @wtroy2
Yes, that is correct. The highest version I had tested it with was 1.2 (not much different from 1.1.6).