Skip to content

Optimize qualimap performance with multi-threaded name sorting #1621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from
Open

Optimize qualimap performance with multi-threaded name sorting #1621

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3d7e090
Optimize qualimap performance with multi-threaded name sorting
pinin4fjords Nov 14, 2025
4406a60
Update changelog for qualimap optimization PR
pinin4fjords Nov 14, 2025
5ee83f8
Disable publishing of intermediate sorted BAM files
pinin4fjords Nov 14, 2025
e481c5a
Update test snapshots for SAMTOOLS_SORT_QUALIMAP
pinin4fjords Nov 14, 2025
3bd27e8
Update version counts in QUALIMAP test blocks
pinin4fjords Nov 14, 2025
c31e770
Final test file fix
pinin4fjords Nov 14, 2025
b9d5e08
Final test file fix
pinin4fjords Nov 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Special thanks to the following for their contributions to the release:

- [PR #1608](https://github.com/nf-core/rnaseq/pull/1608) - Bump version after release 3.21.0
- [PR #1617](https://github.com/nf-core/rnaseq/pull/1617) - Update bbmap/bbsplit module
- [PR #1621](https://github.com/nf-core/rnaseq/pull/1621) - Optimize qualimap performance with multi-threaded name sorting

## [[3.21.0](https://github.com/nf-core/rnaseq/releases/tag/3.21.0)] - 2025-09-18

Expand Down
9 changes: 9 additions & 0 deletions modules/nf-core/qualimap/rnaseq/nextflow.config
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 8 additions & 2 deletions tests/bam_input.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"BAM input for Salmon": {
"content": [
139,
144,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -79,6 +79,9 @@
"SAMTOOLS_INDEX": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down Expand Up @@ -958,7 +961,7 @@
},
"BAM input for RSEM": {
"content": [
138,
143,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -1041,6 +1044,9 @@
"SAMTOOLS_INDEX": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/default.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@
},
"Params: default": {
"content": [
209,
214,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -199,6 +199,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/featurecounts_group_type.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@
},
"Params: --featurecounts_group_type false": {
"content": [
199,
204,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -196,6 +196,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/hisat2.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@
},
"Params: --aligner hisat2": {
"content": [
200,
205,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -203,6 +203,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/min_mapped_reads.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"Params: --min_mapped_reads 90": {
"content": [
163,
166,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -98,6 +98,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/remove_ribo_rna.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"Params: --remove_ribo_rna": {
"content": [
220,
225,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -98,6 +98,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/sentieon_default.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@
},
"Params: default --use_sentieon_star": {
"content": [
209,
214,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -199,6 +199,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/skip_trimming.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"Params: --skip_trimming": {
"content": [
199,
204,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -98,6 +98,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
5 changes: 4 additions & 1 deletion tests/star_rsem.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"Params: --aligner star_rsem": {
"content": [
207,
212,
{
"BBMAP_BBSPLIT": {
"bbmap": 39.18
Expand Down Expand Up @@ -109,6 +109,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
10 changes: 8 additions & 2 deletions tests/umi.nf.test.snap
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"--umi_dedup_tool 'umitools'": {
"content": [
261,
266,
{
"BEDTOOLS_GENOMECOV_FW": {
"bedtools": "2.31.1"
Expand Down Expand Up @@ -92,6 +92,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down Expand Up @@ -1507,7 +1510,7 @@
},
"Params: --aligner hisat2 --umi_dedup_tool 'umicollapse'": {
"content": [
194,
199,
{
"BEDTOOLS_GENOMECOV_FW": {
"bedtools": "2.31.1"
Expand Down Expand Up @@ -1601,6 +1604,9 @@
"SAMTOOLS_SORT": {
"samtools": 1.21
},
"SAMTOOLS_SORT_QUALIMAP": {
"samtools": 1.21
},
"SAMTOOLS_STATS": {
"samtools": 1.21
},
Expand Down
10 changes: 9 additions & 1 deletion workflows/rnaseq/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ include { MULTIQC } from '../../modules/nf-core/multiqc'
include { BEDTOOLS_GENOMECOV as BEDTOOLS_GENOMECOV_FW } from '../../modules/nf-core/bedtools/genomecov'
include { BEDTOOLS_GENOMECOV as BEDTOOLS_GENOMECOV_REV } from '../../modules/nf-core/bedtools/genomecov'
include { SAMTOOLS_INDEX } from '../../modules/nf-core/samtools/index'
include { SAMTOOLS_SORT as SAMTOOLS_SORT_QUALIMAP } from '../../modules/nf-core/samtools/sort'

//
// SUBWORKFLOW: Consisting entirely of nf-core/modules
Expand Down Expand Up @@ -539,8 +540,15 @@ workflow RNASEQ {
//
if (!params.skip_qc) {
if (!params.skip_qualimap) {
QUALIMAP_RNASEQ (
// Sort BAM by name for qualimap (performance optimization)
SAMTOOLS_SORT_QUALIMAP (
ch_genome_bam,
ch_fasta.map { [ [:], it ] }
)
ch_versions = ch_versions.mix(SAMTOOLS_SORT_QUALIMAP.out.versions.first())

QUALIMAP_RNASEQ (
SAMTOOLS_SORT_QUALIMAP.out.bam,
ch_gtf.map { [ [:], it ] }
)
ch_multiqc_files = ch_multiqc_files.mix(QUALIMAP_RNASEQ.out.results.collect{it[1]})
Expand Down
Loading