AbberantExpression module error

[Roshanshafiha]

Hi , sorry for raising many issues , but I am trying to run this pipeline on one of my projects and I am getting stuck.
I have the below error :
Failed to solve scheduling problem with ILP solver in time (10s). Falling back to greedy solver.

[Thu Mar 27 13:25:56 2025]
rule AberrantExpression_pipeline_Counting_countReads_R:
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/bam_files/X07007248510_S6_L001-ready.bam, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/count_ranges.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/params/counts/7007248510_countParams.csv, Scripts/AberrantExpression/pipeline/Counting/countReads.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/counts/7007248510.Rds
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.drop/tmp/AE/v47/counts/7007248510.Rds
jobid: 57
reason: Missing output files: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/counts/7007248510.Rds; Input files updated by another job: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/count_ranges.Rds
wildcards: annotation=v47, sampleID=7007248510
resources: tmpdir=/tmp

[Thu Mar 27 13:33:53 2025]
Error in rule AberrantExpression_pipeline_Counting_countReads_R:
jobid: 137
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/bam_files/X0701298521_S174_L004-ready.bam, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/count_ranges.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/params/counts/701298521_countParams.csv, Scripts/AberrantExpression/pipeline/Counting/countReads.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_data/aberrant_expression/v47/counts/701298521.Rds
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.drop/tmp/AE/v47/counts/701298521.Rds (check log file(s) for error details)

RuleException:
CalledProcessError in file /tmp/tmpzq18yapv, line 34:
Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.snakemake/scripts/tmp_zn92n09.countReads.R' returned non-zero exit status 137.
File "/tmp/tmpzq18yapv", line 34, in __rule_AberrantExpression_pipeline_Counting_countReads_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/concurrent/futures/thread.py", line 59, in run

My whole log file is attached for your reference

2025-03-27T130627.479523.snakemake.log

[AtaJadidAhari]

Hi @Roshanshafiha and thanks for using DROP!

This is usually a memory error. Increase the memory and rerun again. Use --rerun-trigger mtime to not run everything from the beginning and -k to continue running even if some jobs failed.

Failed to solve scheduling problem with ILP solver in time (10s). Falling back to greedy solver. is not an issue in running DROP, probably you are using some scheduler (SLURM?) and it's related to that. Your job is anyways running and the scheduler has successfully started your job.

[Roshanshafiha]

Hi, thank you so much for the update. I am rerunning it with a higher memory now. May I know how long the pipeline (aberrantExpression step alone) would roughly take to complete for 74 samples ?

[Roshanshafiha]

Hi I have another error now , attached below , may I know if this is due to the estimateSizeFactorsForMatrix() error and is this because some of my samples are bad? I have samples with sequencing depth of 20 million per reads and 50 million per reads.

processing file: /tmp/RtmpkjtcD6/file2eb1e31b222f65/Summary.spin.Rmd
Error in estimateSizeFactorsForMatrix():
! every gene contains at least one zero, cannot compute log geometric means
Backtrace:
▆

1. └─OUTRIDER::plotCountGeneSampleHeatmap(...)
2. ├─base::t(t(fcMat)/estimateSizeFactorsForMatrix(fcMat))
3. └─DESeq2::estimateSizeFactorsForMatrix(fcMat)

Quitting from /tmp/RtmpkjtcD6/file2eb1e31b222f65/Summary.spin.Rmd:80-87 [geneSampleHeatmap]
Execution halted
Full Traceback (most recent call last):
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/executors/init.py", line 2656, in run_wrapper
run(
File "/tmp/tmprzaxx7dk", line 232, in __rule_AberrantExpression_pipeline_OUTRIDER_Summary_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 1567, in script
executor.evaluate()
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 455, in evaluate
self.execute_script(fd.name, edit=edit)
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 819, in execute_script
self._execute_cmd("Rscript --vanilla {fname:q}", fname=fname)
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 486, in execute_cmd
return shell(
^^^^^^
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/shell.py", line 294, in new
raise sp.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.snakemake/scripts/tmp8cskgly.wBRender.R' returned non-zero exit status 1.

[Tue Apr 8 14:00:02 2025]
Error in rule AberrantExpression_pipeline_OUTRIDER_Summary_R:
jobid: 161
input: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_results/aberrant_expression/v47/outrider/control/ods.Rds, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/processed_results/aberrant_expression/v47/outrider/control/OUTRIDER_results.tsv, Scripts/AberrantExpression/pipeline/OUTRIDER/Summary.R
output: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/html/AberrantExpression/Outrider/v47/Summary_control.html, /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/Output/html/AberrantExpression/Outrider/v47/OUTRIDER_results_control.tsv
log: /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.drop/tmp/AE/v47/control/OUTRIDER_summary.Rds (check log file(s) for error details)

Full Traceback (most recent call last):
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/executors/init.py", line 2656, in run_wrapper
run(
File "/tmp/tmprzaxx7dk", line 232, in __rule_AberrantExpression_pipeline_OUTRIDER_Summary_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 1567, in script
executor.evaluate()
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 455, in evaluate
self.execute_script(fd.name, edit=edit)
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 819, in execute_script
self._execute_cmd("Rscript --vanilla {fname:q}", fname=fname)
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/script.py", line 486, in execute_cmd
return shell(
^^^^^^
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/shell.py", line 294, in new
raise sp.CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.snakemake/scripts/tmp8cskgly.wBRender.R' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/executors/init.py", line 715, in callback
raise ex
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/executors/init.py", line 699, in cached_or_run
run_func(*args)
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/site-packages/snakemake/executors/init.py", line 2692, in run_wrapper
raise RuleException(
snakemake.exceptions.RuleException: CalledProcessError in file /tmp/tmprzaxx7dk, line 72:
Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.snakemake/scripts/tmp8cskgly.wBRender.R' returned non-zero exit status 1.
File "/tmp/tmprzaxx7dk", line 72, in __rule_AberrantExpression_pipeline_OUTRIDER_Summary_R

RuleException:
CalledProcessError in file /tmp/tmprzaxx7dk, line 72:
Command 'set -euo pipefail; Rscript --vanilla /gpfs/projects/tmedicine/Shafiha/pipelines/DROP/DROP_pipeline/.snakemake/scripts/tmp8cskgly_.wBRender.R' returned non-zero exit status 1.
File "/tmp/tmprzaxx7dk", line 72, in __rule_AberrantExpression_pipeline_OUTRIDER_Summary_R
File "/gpfs/software/genomics/DROP/v1.4.0/lib/python3.12/concurrent/futures/thread.py", line 59, in run
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2025-04-08T135842.621415.snakemake.log
unlocking
removing lock
removing lock
removed all locks

[vyepez88]

Hi, thank you so much for the update. I am rerunning it with a higher memory now. May I know how long the pipeline (aberrantExpression step alone) would roughly take to complete for 74 samples ?

Very much depends on the number of cores provided. A sample takes ~20min to be counted/core. The OUTRIDER fit with 30 cores should take <1h.

[Roshanshafiha]

Hi @AtaJadidAhari , the error related to the estimateSizeFactorsForMatrix(): which I have mentioned above is it due to the sequencing depth difference which my data currently is sequenced under?

[vyepez88]

Hi, it is quite weird that every gene has at least one zero. Can you load the count matrix {root}/processed_data/aberrant_expression/{geneAnnotation}/outrider/{drop_group}/total_counts.Rds and try to explore what happened? Could be that 1 sample has very low reads. Double-check that the strand was correctly assigned for all samples. A common error is to not specify correctly between yes and reverse.
you could low the count matrix and do hist(colSums(count_matrix))

[Roshanshafiha]

Hi @vyepez88 , thanks for the response. May I know what should I be stating , yes or reverse ?

I checked the count matrix and I found there are many zeroes within the gene , which seem relevant to this error : https://www.biostars.org/p/440379/

I modified the count matrix for both my drop group in the location by adding +1 {root}/processed_data/aberrant_expression/{geneAnnotation}/outrider/{drop_group}/total_counts.Rds and re-ran the pipeline, this time it completed without giving any error. I wanted to clarify by adding +1 at a later stage would this advisable or should I avoid this?

[vyepez88]

Hi, sorry, I hadn't seen this.
STRAND: either yes, no, or reverse: ‘no’ means that the sequencing was not strand specific; ‘yes’ that it was strand specific, and the first read in the pair is on the same strand as the feature and the second read on the opposite strand; and ‘reverse’ that the sequencing is strand specific and the first read in the pair is on the opposite strand to the feature and the second read on the same strand.
Adding + 1 is not really solving the root of the problem. You probably either have a global problem with the counting or just in one sample. Can you load the ods object and run: hist(colSums(counts(ods))) ?