Skip to content

Implement outgroup handling#24

Open
gmanthey wants to merge 6 commits intomainfrom
21-add-proper-outgroup-handling
Open

Implement outgroup handling#24
gmanthey wants to merge 6 commits intomainfrom
21-add-proper-outgroup-handling

Conversation

@gmanthey
Copy link
Copy Markdown
Owner

@gmanthey gmanthey commented May 5, 2026

Closes #21

Workflow for outgroup files:

  • treat fastq files until bam the same as other files
  • call only on the set of variants identified after quality/depth filter (this is the first place where vcf file is only variants and not all sites)
  • merge into the all vcf file

- treat fastq files until bam the same
- call only on the set of variants identified after quality/depth filter (this is the first place where vcf file is only variants and not all sites)
- merge into the all vcf file
@gmanthey gmanthey requested a review from matwe340 May 5, 2026 10:15
@gmanthey gmanthey linked an issue May 5, 2026 that may be closed by this pull request
@gmanthey
Copy link
Copy Markdown
Owner Author

gmanthey commented May 5, 2026

Needs testing. (We should consider adding a testing dataset)

@gmanthey gmanthey mentioned this pull request May 5, 2026
@gmanthey
Copy link
Copy Markdown
Owner Author

gmanthey commented May 7, 2026

Tests run through now.

Some caveats/issues here:

  • Currently the outgroup individuals are called parallel on each chromosome, which was tried but removed for all individuals at some point because it creates very many snakemake jobs if there are many individuals and many chromosomes (as they are multiplied), which causes the graph resolution to get slow. With few outgroup individuals, this should not cause huge issues, but might if that number gets higher.
  • The point at which the outgroup individuals are added to the vcf could potentially influence filtering, as it is before the minimum depth filter on SNPs. It could be better to move merging the outgroup in to after this step?

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements explicit outgroup handling in the Snakemake variant-calling pipeline so outgroup individuals are processed through mapping like other samples, then genotyped only at already-identified variant sites and merged into the final VCF.

Changes:

  • Added outgroup_individuals config and updated sample parsing to optionally include/exclude outgroup individuals.
  • Added per-chromosome outgroup calling and a merge step to join outgroup genotypes into chromosome VCFs before downstream filtering/merging.
  • Renamed/propagated final VCF naming to include the outgroup stage (IF-OG-...) and updated example config/inputs.

Reviewed changes

Copilot reviewed 10 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
workflow/Snakefile Updates the default “all” target to the new final VCF including outgroup stage.
workflow/rules/scripts.smk Extends get_individuals() with an include_outgroup toggle driven by config.
workflow/rules/read_processing.smk Includes outgroup individuals in FASTQ-trimming/summary expansions.
workflow/rules/read_mapping.smk Includes outgroup individuals in BAM generation targets.
workflow/rules/call.smk Generalizes per-sample rename inputs via {group} and adds per-chromosome outgroup calling.
workflow/rules/filter.smk Excludes outgroup from initial cohort filtering, then merges outgroup calls into IF VCFs and continues filtering.
workflow/rules/consensus.smk Includes outgroup individuals in consensus FASTA generation.
example/individuals.txt Adds an example outgroup individual FASTQ entries.
example/config.yml Adds outgroup_individuals example configuration.
config.yml.template Adds outgroup_individuals default configuration key.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread workflow/Snakefile
Comment thread workflow/rules/scripts.smk Outdated
Comment thread workflow/rules/filter.smk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add proper outgroup handling

2 participants