Error in rule AberrantExpression_pipeline_Counting_mergeCounts_R

[kdewispelaere]

Hi authors,

Thanks for the amazing package. I ran into a problem with the new DROP-1.4.4 when doing aberrant expression analysis. Error seems to be in the count merging step. Have you encountered a similar problem before and could you help me debug this please?

2053 of 2083 steps (99%) done Select jobs to execute... [Fri Mar 14 13:05:26 2025]
rule AberrantExpression_pipeline_Counting_mergeCounts_R: input: /.../processed_data/aberrant_expression/v110/count_ranges.Rds, /.../processed_data/aberrant_expression/v110/params/merge/Neutrophils_... output: /.../processed_data/aberrant_expression/v110/outrider/Neutrophils/total_counts.Rds log: /.../drop/.drop/tmp/AE/v110/Neutrophils/merge.Rds
jobid: 1044
reason: Missing output files: /.../processed_data/aberrant_expression/v110/outrider/Neutrophils/total_counts.Rds; Input files updated by another job: .../$
wildcards: annotation=v110, dataset=Neutrophils threads: 6
resources: tmpdir=/tmp There were 12 warnings (use warnings() to see them) read 0 files
Error in seq_len(ncol(a1)) : argument must be coercible to non-negative integer Calls: SummarizedExperiment -> DataFrame In addition: Warning message:
In seq_len(ncol(a1)) : first element used of 'length.out' argument
Execution halted
[Fri Mar 14 13:05:34 2025]
Error in rule AberrantExpression_pipeline_Counting_mergeCounts_R: jobid: 1044
output: .../processed_data/aberrant_expression/v110/outrider/Neutrophils/total_counts.Rds log: .../drop/.drop/tmp/AE/v110/Neutrophils/merge.Rds (check log file(s) for error message) RuleException: CalledProcessError in line 57 of /tmp/tmpd59cqyqa: Command 'set -euo pipefail; Rscript --vanilla .../drop/.snakemake/scripts/tmp_0lyrz61.mergeCounts.R' returned non-zero exit status 1.
File "/tmp/tmpd59cqyqa", line 57, in __rule_AberrantExpression_pipeline_Counting_mergeCounts_R
File "/usr/local/software/archive/linux-scientific7-x86_64/gcc-9/miniconda3-4.7.12.1-rmuek6r3f6p3v6fdj7o2klyzta3qhslh/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2025-03-14T130008.462723.snakemake.log

this is an exerpt from the sample annotation:
RNA_ID STRAND PAIRED_END RNA_BAM_FILE DROP_GROUP COUNT_MODE COUNT_OVERLAPS GENE_ANNOTATION GENOME DNA_VCF_FILE DNA_ID
S1 no TRUE /.../....markdup.sorted.bam Platelets Union TRUE v110 ncbi
S2 no TRUE /.../....markdup.sorted.bam Platelets Union TRUE v110 ncbi
S3 no TRUE /.../....markdup.sorted.bam Platelets Union TRUE v110 ncbi
S4 no TRUE /.../....markdup.sorted.bam Platelets Union TRUE v110 ncbi
S5 no TRUE /.../....markdup.sorted.bam Platelets Union TRUE v110 ncbi

Thank you,
Koen

[AtaJadidAhari]

Hi @kdewispelaere, it seems like DROP fails to correctly read the count files for some reason. You can see this from:

resources: tmpdir=/tmp There were 12 warnings (use warnings() to see them) read 0 files

stating that it read in 0 files, when it should read in all the counts which belong to that DROP group. Hard to say why this happens. Can you make sure the counts Rds files exist? They should be located here: processed_data/aberrant_expression/{annotation}/counts/{sample_id}.Rds

Also try to read one in and check the content to see if it is correct by running readRDS(file_path).

[kdewispelaere]

processed_data/aberrant_expression/{annotation}/counts/ actually doesn't even exist, count_ranges.Rds and a folder 'params' containing 'counts', 'merge' and 'results' of all samples are created

[AtaJadidAhari]

If the count RDS files are not there, probably something went wrong in the counting step. Can you share the output of snakemake aberrantExpression -n --rerun-trigger mtime?

[kdewispelaere]

Hi,
Thanks for the reply! Here's the output of the dry run:

output.txt

[vyepez88]

Hi, sorry for the super late reply. Were you able to sort this out? If not, can you try running:

snakemake aberrantExpression -nF --rerun-trigger mtime