Skip to content

Optimize qualimap performance with multi-threaded name sorting #1621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: dev
Choose a base branch
from
Open

Optimize qualimap performance with multi-threaded name sorting #1621

wants to merge 7 commits into from

Conversation

@pinin4fjords
Copy link
Member

@pinin4fjords pinin4fjords commented Nov 14, 2025

Summary

This PR optimizes the qualimap rnaseq step by pre-sorting BAM files using multi-threaded samtools before running qualimap with the --sorted flag.

Changes

  • Added SAMTOOLS_SORT_QUALIMAP process before QUALIMAP_RNASEQ in the workflow
  • Configured samtools sort to use -n flag for name-based sorting with multi-threading
  • Updated qualimap to use --sorted flag with pre-sorted BAM files

Performance Impact

Based on the original issue report, this optimization reduces total execution time from ~85 minutes to under 40 minutes:

  • Qualimap (single-threaded sorting): ~85 minutes
  • New approach: Samtools sort (multi-threaded): ~7 min + Qualimap (pre-sorted): ~33 min = ~40 minutes total

This represents a >50% reduction in execution time for the qualimap step.

Testing

  • Syntax validated
  • Ready for pipeline testing

Resolves #1356

🤖 Generated with Claude Code

pinin4fjords and others added 2 commits November 14, 2025 12:48
@pinin4fjords @claude
Optimize qualimap performance with multi-threaded name sorting
3d7e090 
Pre-sort BAM files by read name using samtools with multi-threading before
running qualimap rnaseq. This reduces qualimap execution time by >50%
(from ~85 min to <40 min) by offloading the single-threaded sorting to
the multi-threaded samtools sort with the -n flag, then passing the
sorted BAM to qualimap with the --sorted flag.

Resolves #1356

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@pinin4fjords @claude
Update changelog for qualimap optimization PR
4406a60 
Add changelog entry for PR #1621 which optimizes qualimap performance
through multi-threaded name sorting.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@pinin4fjords pinin4fjords linked an issue Nov 14, 2025 that may be closed by this pull request
Improve qualimap execution time #1356
Open
@pinin4fjords pinin4fjords mentioned this pull request Nov 14, 2025
Open
@github-actions
Copy link

github-actions bot commented Nov 14, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit b9d5e08

+| ✅ 286 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   9 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: assets/multiqc_config.yml
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_if_empty_null - ifEmpty(null) found in /home/runner/work/rnaseq/rnaseq/subworkflows/local/prepare_genome/main.nf: _ versions = ch_versions.ifEmpty(null) // channel: [ versions.yml ]
    _

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 3.4.1
  • Run at 2025-11-14 18:35:22

@pinin4fjords @claude
Disable publishing of intermediate sorted BAM files
5ee83f8 
Prevent the name-sorted BAM files created for qualimap from being
published as they are only intermediate files used for performance
optimization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
pinin4fjords and others added 2 commits November 14, 2025 16:21
@pinin4fjords @claude
Update test snapshots for SAMTOOLS_SORT_QUALIMAP
e481c5a 
Add SAMTOOLS_SORT_QUALIMAP version entry to all test snapshots that
run qualimap, reflecting the new samtools sort step added for
performance optimization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@pinin4fjords @claude
Update version counts in QUALIMAP test blocks
3bd27e8 
Increment version counts by 5 in test blocks that run qualimap to
account for the additional SAMTOOLS_SORT_QUALIMAP process. Only
test blocks containing QUALIMAP_RNASEQ are updated.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@pinin4fjords pinin4fjords added this to the 3.22.0 milestone Nov 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MatthiasZepper MatthiasZepper Awaiting requested review from MatthiasZepper

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

3.22.0

Development

Successfully merging this pull request may close these issues.

Improve qualimap execution time

2 participants

@pinin4fjords