Genotypes QC workflows enhancement

[dmccormi]

Genotype QC to do

I'm (@dmccormi ) submitting this as a single issue as each point requires the context of the other points.

1. QCed genotypes will not just be used for GWAS.

• The pipeline is almost entirely GWAS focused in its approach. Please separate out the genotype QC and GWAS parts of the pipeline into separate pipelines.
• Genotype QC can mostly remain as is for the combine genetic variants part (although please see the Relatedness checks point below).
• I will handle the imputation upload etc.
• Please restart the pipeline after imputation, taking the outputs of topmed imputation as input.
• The output of the Genotype QC pipeline should be the complete set of imputed, QCed genotypes as well as a table of the related individuals for export filtering (see below).
• There are essential steps, dictated by our Information Governance principles, that need to be made between genotype QC and downstream analyses. The outputs of the genotype QC pipeline will feed through this process, with project-specific exports made at the end which can then be put into the downstream GWAS pipeline, in your case. Covariates are generated during these steps; an export will contain both the data and the covariates.

2. Relatedness checks

• Removing all related individuals during the genotype QC steps only makes sense if everyone is going to be analysed together.
• We have many different phenotypes; we may end up removing related individuals where, for example, one is in the COVID cohort and another in the hepatitis cohort. In addition, the lab does many analyses other than GWAS for which we often need genotypes - removing a genotype from a small set because an individual is related to someone with a completely different phenotype in a different analysis should not be done.
• The relatedness output should generate a table / matrix which shows which genotypes must not be used together in a single analysis. I will then use this table / matrix to filter genotypes during the export process (which is project-directed).
• If this will be too difficult, then I propose we move the relatedness step from the Genotype QC pipeline to the more downstream project export step, which will then only remove related individuals in a project subset.

3. Array inputs

• Some of our collaborators (and our other open-source inputs) will / do use other, different types of arrays than our core GenOMICC UK arrays.
• It would be more useful if the Genotype QC pipeline could take a list or table of array paths to pass into the Combine Genetic Variants step.

4. Resilience of pipeline

• Once the pipeline is finished it needs to be modified to be coded in a commonly used language, preferably Python.

[olivierlabayle]

Hi Dominique. As I understand it, you would like to reuse most of the components in the CombineGeneticDatasets and Imputation workflows for generic QC processing of genotypes.

1. Array inputs: Yes good idea.
2. Relatedness: Probably ok as well. However if we do so we also need to separate (at least) the ancestry estimation part since virtually all statistical methods assume independence of the samples which is false when individuals are closely related.
3. Programming language: I have used Julia to glue things together because it is the language I am mostly used to now. This is an easy to use high level language but I don't have a strong opinion if someone wants to take the trouble to rewrite all of it in Python.

Regarding 1. and 2. I can't make it a priority right now but can provide guidance to whomever wants to contribute to the enhancement of the project.

[dmccormi]

Hi Olivier - that sounds fine. I propose then that we split the pipeline at this stage into a QC part, which I am happy to continue development on, and the downstream GWAS part which it looks like you will continue to work on. I will likely move the code over to Python just because I don't have time to learn Julia.

I am currently thinking it would definitely be best to do the relatedness calling just before a project is exported to a user for analysis, taking the phneotype into consideration. So in the case of the All COVID GWAS project you are working on I would select the imputed genotypes of all the (still consented) COVID patients and controls, then run the relatedness filtering step, and then hand over the fully QCed (and independent) dataset to you for GWAS.

In the case of the All GWAS you are planning, we can discuss. We can either break this up into smaller exports by phenotype, or I can just put them all together and do the relatedness filtering before handing over.

[olivierlabayle]

No problem.

I don't know if there is much to be gained to extract unrelated individuals per infection group, it is probably easier to do it only once.

For covariates and phenotypes, let's discuss indeed. My current thinking is that we should always output a consistent table with a variable number of columns:

FID, IID, AGE, SEX, $INFECTION_NAME, SEVERE_$INFECTION_NAME, $DEATH, ...

where the $ columns will be per project. For example:

• The covid study will have only: COVID_19 and SEVERE_COVID_19
• The all GWAS will have: all infections and severity of infection
• The mortality all GWAS will also include death

Please contribute to that repository (on a new git branch), I think this is where it belongs. I am happy to review a PR, ideally smaller changes are easier to review.

[olivierlabayle]

Re Programming language - I am just thinking that given the amount of source code and testing logic implemented in the repo you will have to understand the Julia code to make sure everything is preserved. It might be less time consuming and destructive to learn some Julia rather than reimplementing in Python.