IntronQuantificationStep fails with soybean genome

[chessleensingh]

This is the error message i got

(beers2_env) (base) sachleen-singh@sachleensingh:~/work/Python_Envs/beers_/CAMPAREE$ bin/run_camparee.py -c /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/soybean_expression.yaml -r 9
Running CAMPAREE using the serial job scheduler.
With the following default scheduler parameters:
-default_memory_in_mb : 32000
-default_submission_args : None
-default_num_processors : 8
And a maximum job resubmission limit of 3.
Execution of the Expression Pipeline Started...
Running jobs:0 | Pending jobs:10 | Resub jobs:0 | Completed jobs:0
--Check 10 pending jobs for satisfied dependencies:
Submitting GenomeAlignmentStep command to serial for sample Control_Rep1.
Finished submitting GenomeAlignmentStep command to serial for sample Control_Rep1.
Submitting GenomeAlignmentStep command to serial for sample Treatment_Rep1.
Finished submitting GenomeAlignmentStep command to serial for sample Treatment_Rep1.
Running jobs:0 | Pending jobs:8 | Resub jobs:0 | Completed jobs:2
--Check 8 pending jobs for satisfied dependencies:
Submitting GenomeBamIndexStep command to serial for sample Control_Rep1.
Finished submitting GenomeBamIndexStep command to serial for sample Control_Rep1.
Submitting GenomeBamIndexStep command to serial for sample Treatment_Rep1.
Finished submitting GenomeBamIndexStep command to serial for sample Treatment_Rep1.
Running jobs:0 | Pending jobs:6 | Resub jobs:0 | Completed jobs:4
--Check 6 pending jobs for satisfied dependencies:
Submitting VariantsFinderStep command to serial for sample Control_Rep1.
Finished submitting VariantsFinderStep command to serial for sample Control_Rep1.
Submitting VariantsFinderStep command to serial for sample Treatment_Rep1.
Finished submitting VariantsFinderStep command to serial for sample Treatment_Rep1.
Submitting IntronQuantificationStep command to serial for sample Control_Rep1.
Job submission failed for IntronQuantificationStep:
Job sample: Control_Rep1
Scheduler parameters: {'job_name': 'IntronQuantificationStep_sample1_Control_Rep1', 'stdout_logfile': '/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/logs/sample1/IntronQuantificationStep.serial.out', 'stderr_logfile': '/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/logs/sample1/IntronQuantificationStep.serial.err', 'num_processors': None, 'memory_in_mb': None, 'additional_args': ''}
Job command: python /home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/intron_quant.py --log_directory_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/logs --data_directory_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/data --bam_file /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/data/sample1/genome_alignment.Aligned.sortedByCoord.out.bam --output_directory /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/results_agat/run_9/CAMPAREE/data/sample1 --info_file /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/soybean_Gmax_508_v4_fixed_agat/soybean_Gmax_508_v4_fixed_agat.annotation.txt --par

---

this is the intron quantification.err file
/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/intron_quant.py:91: SyntaxWarning: "is" with a literal. Did you mean "=="?
antisense_strand = "-" if strand is "+" else "+"
Traceback (most recent call last):
File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/intron_quant.py", line 409, in
sys.exit(IntronQuantificationStep.main())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/intron_quant.py", line 388, in main
intron_quant.execute(args.bam_file, args.output_directory, args.info_file)
File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/intron_quant.py", line 50, in execute
self.info = AnnotationInfo(geneinfo_file_path, chrom_lengths)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/annotation_info.py", line 219, in init
mintron_before = mintrons[index_before]
~~~~~~~~^^^^^^^^^^^^^^
IndexError: deque index out of range

please help.

[chessleensingh]

i made sure that i used the right gtf file, i used cufflinks and agat, to make the gtf from gff3, and made genome with star provided with the code, and both times this is where the run stopped.

[brainfood]

Thanks for your interest! Looking at the error, it seems like camparee's running into problems with the annotation file. Judging from your log files, the annotation file appears to be in the correct format, but I want to eliminate that as the source of the problem. Would you please confirm that you used the 'bin/format_reference_files_for_camparee.py' script to convert the GTF file into the tab-delimited format camparee requires?

Is the gff3 file you used as the original annotation available from a public repository? Thanks for your help.

[chessleensingh]

Yes, I used the bin/format_reference_files_for_camparee.py to convert. The gff3 file was indeed downloaded from phytozome. it is the a4 version of the soybean genome. I did not make the star index with the output text file but with the gtf instead (I assume this is not the problem but just to make sure)
thanks for the prompt response.

[brainfood]

I was able to successfully run CAMPAREE on Glycine Max data and didn't see the same error you encountered. This feels like it could be a difference in annotation files. Would you please post the first 10 lines of the gtf file you used, taking care to include a few lines that have "exon" in the third column?

Below I list various details for my test run, for comparison purposes. Please take a look and let me know if you notice any major differences from what you did. That should help us further pinpoint the problem. Thanks for your help and patience.

---

I downloaded the genome sequence and annotation for Gmax_Wm82_a4_v1 from Phytozome. These are the two files I downloaded:

• Genome sequence: Gmax_508_v4.0.fa.gz
• Annotation: Gmax_508_Wm82.a4.v1.gene_exons.gff3.gz

Are these the same files you used?

Next, I used the format_reference_files_for_camparee.py script packaged with CAMPAREE to prepare both the genome sequences and annotation file for CAMPAREE, and to generate the chromosome ploidy file. Here's the command (file paths removed for clarity):

format_reference_files_for_camparee.py -n GlycineMax_Wm82-a4-v1_Phytozome13 \
                                       -g Gmax_508_v4.0.fa.gz \
                                       -a prepped-for-camparee_Gmax_508_Wm82.a4.v1.gene_exons.gff3 \
                                       --create_chr_ploidy_file

Note, I needed to tweak the last column in the gff3 file (semi-colon delimited list of attributes) so it included a 'gene_id' and 'transcript_id' tag. I'll go into more detail about this in subsequent comments, depending on how your gtf looks, compared to the one I generated.

Then I built the STAR index from the genome FASTA, without using any annotation info:

third_party_software/STAR \
    --runMode genomeGenerate \
    --runThreadN 2 \
    --limitGenomeGenerateRAM 12000000000 \
    --genomeDir star_index.genome \
    --outFileNamePrefix star_index.genome/ \
    --genomeFastaFiles GlycineMax_Wm82-a4-v1_Phytozome13.oneline_seqs.fa

For test data, I downloaded FASTQ files for two Glycine max RNA-Seq samples from GEO:

• SRR29070723: Soybean W05 root uninfected control replicate 1
• SRR29070722: Soybean W05 root uninfected control replicate 2
  To speedup runtime, I only used the first 1,000,000 lines from each file (250,000 paired-end reads per sample).

Lastly, I ran CAMPAREE using all these data:
bin/run_camparee.py -c config/gmax_bugfix_short_fastq.config.yaml --debug -r 1
We can compare the contents of our config files later if the annotation files aren't the problem.

I built and ran CAMPAREE with python v3.10.12.

[chessleensingh]

hi,
i tried to make the index without the annotation, that worked, but it still stopped at the intron classification step.

-----------------------------------------GTF_FILE-------------------------first 30 lines------------------------------------------------
##gtf-version X

GFF-like GTF i.e. not checked against any GTF specification. Conversion based on GFF input, standardised by AGAT.

##annot-version Wm82.a4.v1
##species Glycine max
Gm01 phytozomev13 gene 27344 28430 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; ID "Glyma.01G000100.Wm82.a4.v1"; Name "Glyma.01G000100"; ancestorIdentifier "Glyma.01G000100.Wm82.a2.v1";
Gm01 phytozomev13 mRNA 27344 28430 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1"; Name "Glyma.01G000100.1"; Parent "Glyma.01G000100.Wm82.a4.v1"; ancestorIdentifier "Glyma.01G000100.1.Wm82.a2.v1"; longest "1"; pacid "41118016";
Gm01 AGAT exon 27344 27824 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "agat-exon-1"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 AGAT exon 27926 27991 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "agat-exon-2"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 AGAT exon 28139 28430 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "agat-exon-3"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 CDS 27656 27824 . - 1 gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1.CDS.3"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 CDS 27926 27991 . - 1 gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1.CDS.2"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 CDS 28139 28218 . - 0 gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1.CDS.1"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 five_prime_UTR 28219 28430 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1.five_prime_UTR.1"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 three_prime_UTR 27344 27655 . - . gene_id "Glyma.01G000100.Wm82.a4.v1"; transcript_id "Glyma.01G000100.1.Wm82.a4.v1"; ID "Glyma.01G000100.1.Wm82.a4.v1.three_prime_UTR.1"; Parent "Glyma.01G000100.1.Wm82.a4.v1"; pacid "41118016";
Gm01 phytozomev13 gene 52485 52601 . - . gene_id "Glyma.01G000137.Wm82.a4.v1"; ID "Glyma.01G000137.Wm82.a4.v1"; Name "Glyma.01G000137";
Gm01 phytozomev13 mRNA 52485 52601 . - . gene_id "Glyma.01G000137.Wm82.a4.v1"; transcript_id "Glyma.01G000137.1.Wm82.a4.v1"; ID "Glyma.01G000137.1.Wm82.a4.v1"; Name "Glyma.01G000137.1"; Parent "Glyma.01G000137.Wm82.a4.v1"; longest "1"; pacid "41114628";
Gm01 AGAT exon 52485 52601 . - . gene_id "Glyma.01G000137.Wm82.a4.v1"; transcript_id "Glyma.01G000137.1.Wm82.a4.v1"; ID "agat-exon-4"; Parent "Glyma.01G000137.1.Wm82.a4.v1"; pacid "41114628";
Gm01 phytozomev13 CDS 52485 52601 . - 0 gene_id "Glyma.01G000137.Wm82.a4.v1"; transcript_id "Glyma.01G000137.1.Wm82.a4.v1"; ID "Glyma.01G000137.1.Wm82.a4.v1.CDS.1"; Parent "Glyma.01G000137.1.Wm82.a4.v1"; pacid "41114628";
Gm01 phytozomev13 gene 54356 55224 . - . gene_id "Glyma.01G000174.Wm82.a4.v1"; ID "Glyma.01G000174.Wm82.a4.v1"; Name "Glyma.01G000174";
Gm01 phytozomev13 mRNA 54356 55224 . - . gene_id "Glyma.01G000174.Wm82.a4.v1"; transcript_id "Glyma.01G000174.1.Wm82.a4.v1"; ID "Glyma.01G000174.1.Wm82.a4.v1"; Name "Glyma.01G000174.1"; Parent "Glyma.01G000174.Wm82.a4.v1"; longest "1"; pacid "41115343";
Gm01 AGAT exon 54356 54579 . - . gene_id "Glyma.01G000174.Wm82.a4.v1"; transcript_id "Glyma.01G000174.1.Wm82.a4.v1"; ID "agat-exon-5"; Parent "Glyma.01G000174.1.Wm82.a4.v1"; pacid "41115343";
Gm01 AGAT exon 55026 55224 . - . gene_id "Glyma.01G000174.Wm82.a4.v1"; transcript_id "Glyma.01G000174.1.Wm82.a4.v1"; ID "agat-exon-6"; Parent "Glyma.01G000174.1.Wm82.a4.v1"; pacid "41115343";
Gm01 phytozomev13 CDS 54356 54579 . - 2 gene_id "Glyma.01G000174.Wm82.a4.v1"; transcript_id "Glyma.01G000174.1.Wm82.a4.v1"; ID "Glyma.01G000174.1.Wm82.a4.v1.CDS.2"; Parent "Glyma.01G000174.1.Wm82.a4.v1"; pacid "41115343";
Gm01 phytozomev13 CDS 55026 55224 . - 0 gene_id "Glyma.01G000174.Wm82.a4.v1"; transcript_id "Glyma.01G000174.1.Wm82.a4.v1"; ID "Glyma.01G000174.1.Wm82.a4.v1.CDS.1"; Parent "Glyma.01G000174.1.Wm82.a4.v1"; pacid "41115343";
Gm01 phytozomev13 gene 56132 64061 . - . gene_id "Glyma.01G000211.Wm82.a4.v1"; ID "Glyma.01G000211.Wm82.a4.v1"; Name "Glyma.01G000211";
Gm01 phytozomev13 mRNA 56132 64061 . - . gene_id "Glyma.01G000211.Wm82.a4.v1"; transcript_id "Glyma.01G000211.1.Wm82.a4.v1"; ID "Glyma.01G000211.1.Wm82.a4.v1"; Name "Glyma.01G000211.1"; Parent "Glyma.01G000211.Wm82.a4.v1"; longest "1"; pacid "41114854";
Gm01 AGAT exon 56132 56139 . - . gene_id "Glyma.01G000211.Wm82.a4.v1"; transcript_id "Glyma.01G000211.1.Wm82.a4.v1"; ID "agat-exon-7"; Parent "Glyma.01G000211.1.Wm82.a4.v1"; pacid "41114854";
Gm01 AGAT exon 63085 63417 . - . gene_id "Glyma.01G000211.Wm82.a4.v1"; transcript_id "Glyma.01G000211.1.Wm82.a4.v1"; ID "agat-exon-8"; Parent "Glyma.01G000211.1.Wm82.a4.v1"; pacid "41114854";
Gm01 AGAT exon 64055 64061 . - . gene_id "Glyma.01G000211.Wm82.a4.v1"; transcript_id "Glyma.01G000211.1.Wm82.a4.v1"; ID "agat-exon-9"; Parent "Glyma.01G000211.1.Wm82.a4.v1"; pacid "41114854";
Gm01 phytozomev13 CDS 56132 56139 . - 2 gene_id "Glyma.01G000211.Wm82.a4.v1"; transcript_id "Glyma.01G000211.1.Wm82.a4.v1"; ID "Glyma.01G000211.1.Wm82.a4.v1.CDS.3"; Parent "Glyma.01G000211.1.Wm82.a4.v1"; pacid "41114854";

[brainfood]

I think I've isolated the problem. It looks like AGAT changes the order of exons for minus-strand transcripts. Here are the exons for the first transcript in the AGAT output:

Gm01	AGAT	exon	27344	27824	.	-	.	gene_id "Glyma.01G000100.Wm82.a4.v1"; ...
Gm01	AGAT	exon	27926	27991	.	-	.	gene_id "Glyma.01G000100.Wm82.a4.v1"; ...
Gm01	AGAT	exon	28139	28430	.	-	.	gene_id "Glyma.01G000100.Wm82.a4.v1"; ...

And here are the same exons from the GTF file I've been using:

Gm01	phytozomev13	exon	28139	28430	.	-	.	gene_id "Glyma.01G000100";...
Gm01	phytozomev13	exon	27926	27991	.	-	.	gene_id "Glyma.01G000100";...
Gm01	phytozomev13	exon	27344	27824	.	-	.	gene_id "Glyma.01G000100";...

AGAT sorts the exons by genomic coordinate, while the original GTF/GFF file lists them in the order they occur in the transcript. These orders are the same for plus-strand transcripts, but are reversed for minus strand transcripts. The CAMPAREE script for processing these files and creating the annotation file assumes the exons are listed in transcript order, rather than by genomic coordinate. I've applied a fix for this to the script so it can work with exons in any order. I'm currently testing the fix and confirming the exon order is the cause of the error you originally encountered. I'll let you know once everything is ready to go.

I should have this ready in the next day or two. In the event you want to get going immediately, here's some command-line perl that you can run instead of AGAT. It adds the 'gene_id' and 'transcript_id' attributes to the GFF3 output so it's ready for use with the current version of CAMPAREE's format_reference_files_for_camparee.py script:

cat /path/to/phytozome/gtf/Gmax_508_Wm82.a4.v1.gene_exons.gff3 | \
perl -ne 'chomp();
if($_ !~ m/^\#/) {
    @line_data = split("\t", $_);
    $attributes = $line_data[8];
    $attributes =~ m/ID=(Glyma\.[^\.]+)(\.\d+)?\./;
    $gene_id = $1;
    $tx_suffix = $2;
    $transcript_tag = "";
    if($tx_suffix ne "") {
        $transcript_tag = "transcript_id \"$gene_id$tx_suffix\";";
    }
    $attributes = "gene_id \"$gene_id\";$transcript_tag$attributes";
    $line_data[8] = $attributes;
    print join("\t", @line_data) . "\n";
} else {
    print "$_\n";
}
' > prepped-for-camparee_Gmax_508_Wm82.a4.v1.gene_exons.gff3

Thanks for your continued help and patience!

[brainfood]

Unfortunately, I wasn't able to get the same error as you using the snippet of the GTF file you provided. We can see if the fix to the 'format_reference_files_for_camparee.py' solves the problem. Would you be willing to download the version of CAMPAREE in the 'fix_intronquant_bug_w_soybean_genome' branch (commit 6a12f07), and try it with your data? You would need to re-run the 'format_reference_files_for_camparee.py' script to reconstruct the annotation file from your gff. Thanks.

[chessleensingh]

I think I can download and run the version of camparee.
thank you for promptly helping me out.

[chessleensingh]

it finished without errror. :) thank you so much

[brainfood]

Great! Thanks for helping us find this bug and test the fix.

[chessleensingh]

This should ideally go into beers and work? I did run it with beers and it ran completely but the output files had no data. what all do i need to share so that you can replicate it.

[tgbrooks]

Hi Chessleensign,

I think the most common reason to end up with a completed BEERS2 run but no output data would be:

• Didn't include polyA tails on the input data. The config file templates for BEERS2 specify a polyA selection step, which depends upon the molecules having polyA tails. These are typically added by CAMPAREE automatically but you could double check the molecules or try removing the polyA selection step from the BEERS2 config.
• Not providing enough input material to BEERS2. Each input molecule can end up as zero, one, or many output reads (due to effects like PCR, fragmentation, size selection, etc.). So if you provide very few input molecules, or molecules that are difficult to sequence (i.e., that BEERS2 models as unlikely to be sequenced), then there is a chance that you get no output reads. When running from CAMPAREE distributions, you can specify the number of molecules per packet and the number of packets to generate in the BEERS2 configuration file, so try increasing those numbers.
• Misspecifying the input directories so that BEERS2 doesn't have any output to work with. I don't recall whether this gives an error or not.
• Something else in the configuration eliminating all your molecules (e.g., size selection if you're trying to get small RNAs)

If you look through the library preparation logs, you should see reports after each step of the number of molecules by size, so you should be able to see whether BEERS2 has anything to work with or for which step they all go to zero.

[chessleensingh]

@tgbrooks , It seemed to work once i increased the molecule numbers to use, thanks.
@brainfood I do have another question for you though,
I 'm running into another error.

here is my terminal output

(beers2_env) (base) sachleen-singh@sachleensingh:~/work/Python_Envs/beers_/fresh_soybean$ ./run_sansomeana_data.sh 
Starting P. sansomeana simulation pipeline at Thu May 15 09:59:47 AM CDT 2025
Log file: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/logs/sansomeana_simulation_20250515_095947.log
----------------------------------------
Found input file: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655578_1.fastq.gz (2.8G)
Found input file: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655578_2.fastq.gz (2.9G)
Found input file: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655579_1.fastq.gz (2.3G)
Found input file: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655579_2.fastq.gz (2.4G)
Step 1: Running CAMPAREE with config: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana.camparee.config.yaml
Starting at: Thu May 15 09:59:47 AM CDT 2025
Running CAMPAREE using the serial job scheduler.
With the following default scheduler parameters:
 	-default_num_processors : 8
	-default_submission_args : None
	-default_memory_in_mb : 32000
Job submission failed for GenomeBuilderStep:
    Job sample: Control_Williams82
    Scheduler parameters: {'job_name': 'GenomeBuilderStep_sample1_Control_Williams82', 'stdout_logfile': '/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs/sample1/GenomeBuilderStep.serial.out', 'stderr_logfile': '/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs/sample1/GenomeBuilderStep.serial.err', 'num_processors': None, 'memory_in_mb': None, 'additional_args': ''}
    Job command:  python /home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/genome_builder.py --log_directory_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs --data_directory_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data --sample 'Sample(sample_id="1",  sample_name="Control_Williams82", fastq_file_paths=["/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655579_1.fastq.gz","/home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/SRR25655579_2.fastq.gz"], pooled="False", bam_file_path="", adapter_sequences=[""], gender="male", molecule_count="50000")' --phased_vcf_file_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/beagle.vcf.gz --chr_ploidy_file_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/GlycineMax_Wm82-a4-v1_updated/GlycineMax_Wm82-a4-v1_updated.chr_ploidy.txt --reference_genome_file_path /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/GlycineMax_Wm82-a4-v1_updated/GlycineMax_Wm82-a4-v1_updated.oneline_seqs.fa
And a maximum job resubmission limit of 3.
Execution of the Expression Pipeline Started...
Running jobs:0 | Pending jobs:10 | Resub jobs:0 | Completed jobs:0
--Check 10 pending jobs for satisfied dependencies:
	Submitting GenomeAlignmentStep command to serial for sample Control_Williams82.
	Finished submitting GenomeAlignmentStep command to serial for sample Control_Williams82.
	Submitting GenomeAlignmentStep command to serial for sample Treatment_Williams82.
	Finished submitting GenomeAlignmentStep command to serial for sample Treatment_Williams82.
Running jobs:0 | Pending jobs:8 | Resub jobs:0 | Completed jobs:2
--Check 8 pending jobs for satisfied dependencies:
	Submitting GenomeBamIndexStep command to serial for sample Control_Williams82.
	Finished submitting GenomeBamIndexStep command to serial for sample Control_Williams82.
	Submitting GenomeBamIndexStep command to serial for sample Treatment_Williams82.
	Finished submitting GenomeBamIndexStep command to serial for sample Treatment_Williams82.
Running jobs:0 | Pending jobs:6 | Resub jobs:0 | Completed jobs:4
--Check 6 pending jobs for satisfied dependencies:
	Submitting VariantsFinderStep command to serial for sample Control_Williams82.
	Finished submitting VariantsFinderStep command to serial for sample Control_Williams82.
	Submitting VariantsFinderStep command to serial for sample Treatment_Williams82.
	Finished submitting VariantsFinderStep command to serial for sample Treatment_Williams82.
	Submitting IntronQuantificationStep command to serial for sample Control_Williams82.
	Finished submitting IntronQuantificationStep command to serial for sample Control_Williams82.
	Submitting IntronQuantificationStep command to serial for sample Treatment_Williams82.
	Finished submitting IntronQuantificationStep command to serial for sample Treatment_Williams82.
Running jobs:0 | Pending jobs:2 | Resub jobs:0 | Completed jobs:8
--Check 2 pending jobs for satisfied dependencies:
	Submitting VariantsCompilationStep command to serial.
	Finished submitting VariantsCompilationStep command to serial.
Running jobs:0 | Pending jobs:1 | Resub jobs:0 | Completed jobs:9
--Check 1 pending jobs for satisfied dependencies:
	Submitting BeagleStep command to serial.
	Finished submitting BeagleStep command to serial.
Running jobs:0 | Pending jobs:0 | Resub jobs:0 | Completed jobs:10
Processing sample1 (Control_Williams82)...
Processing sample2 (Treatment_Williams82)...
Running jobs:0 | Pending jobs:32 | Resub jobs:0 | Completed jobs:10
--Check 32 pending jobs for satisfied dependencies:
	Submitting GenomeBuilderStep command to serial for sample Control_Williams82.
Traceback (most recent call last):
  File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/bin/run_camparee.py", line 25, in <module>
    args.func(args)
  File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/camparee_controller.py", line 68, in run_camparee_pipeline
    ExpressionPipeline.main(self.configuration, scheduler_mode,
  File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/expression_pipeline.py", line 747, in main
    pipeline.execute()
  File "/home/sachleen-singh/work/Python_Envs/beers_/CAMPAREE/camparee/expression_pipeline.py", line 649, in execute
    self.expression_pipeline_monitor.monitor_until_all_jobs_completed(queue_update_interval=10)
  File "/home/sachleen-singh/work/Python_Envs/beers_/beers2_env/lib/python3.11/site-packages/beers_utils/job_monitor.py", line 242, in monitor_until_all_jobs_completed
    self.submit_pending_job(pend_job_id)
  File "/home/sachleen-singh/work/Python_Envs/beers_/beers2_env/lib/python3.11/site-packages/beers_utils/job_monitor.py", line 391, in submit_pending_job
    raise JobMonitorException(f"Job submission failed for {job.step_name}. "
beers_utils.job_monitor.JobMonitorException: Job submission failed for GenomeBuilderStep. See log file for full details.
CAMPAREE completed at: Thu May 15 11:25:23 AM CDT 2025
----------------------------------------
Step 2: Running BEERS with config: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana.beers.config.yaml
Starting at: Thu May 15 11:25:23 AM CDT 2025
Building DAG of jobs...
MissingInputException in rule run_library_prep_packet_from_distribution in file /home/sachleen-singh/work/Python_Envs/beers_/beers2_env/lib/python3.11/site-packages/beers/Snakefile, line 125:
Missing input files for rule run_library_prep_packet_from_distribution:
    output: library_prep_pipeline/sample1/from_distribution/library_prep_pipeline_result_molecule_pkt0.txt, library_prep_pipeline/sample1/from_distribution/library_prep_pipeline_result_molecule_pkt0.quant_file, library_prep_pipeline/sample1/from_distribution/input_quants/library_prep_pipeline_molecule_pkt0.quant_file, library_prep_pipeline/sample1/logs/PolyAStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/FragmentStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/FirstStrandSynthesisStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/SecondStrandSynthesisStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/SizingStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/AdapterLigationStep/molecule_pkt0.log, library_prep_pipeline/sample1/logs/PCRAmplificationStep/molecule_pkt0.log
    wildcards: sample=1, packet_num=0
    affected files:
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/updated_annotation_1.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/isoform_psi_value_quantifications.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/custom_genome_2.fa
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/custom_genome_indels_1.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/updated_annotation_2.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/custom_genome_1.fa
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/custom_genome_indels_2.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/gene_quantifications.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/allelic_imbalance_quantifications.txt
        /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/data/Control_Williams82/intron_quantifications.txt
BEERS completed at: Thu May 15 11:25:24 AM CDT 2025
----------------------------------------
P. sansomeana simulation pipeline completed successfully at Thu May 15 11:25:24 AM CDT 2025
Output located in: /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/
(beers2_env) (base) sachleen-singh@sachleensingh:~/work/Python_Envs/beers_/fresh_soybean$ 

seems like I'm running into Genome building error, but I made sure there is enough memory. I'm still at a loss why is it not working.

[brainfood]

Hi chessleensingh,

I have a few questions that should (hopefully) help us identify the problem. Was this CAMPAREE run configured in the same way as one that worked for you previously? If not, how does it differ from the one that previously worked (i.e. different annotation files, input files, config options, CAMPAREE installation, etc.)?

Would you please share the contents of the stdout, stderr, and general log files for the failed step? Based on what you shared above, they should be located here:

• /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs/sample1/GenomeBuilderStep.serial.out
• /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs/sample1/GenomeBuilderStep.serial.err
• /home/sachleen-singh/work/Python_Envs/beers_/fresh_soybean/sansomeana/run_9/CAMPAREE/logs/sample1/GenomeBuilderStep.log

Thanks.