Protein function prediction with GO

[sfluegel05]

Until now, we have only used our framework for ChEBI, but in principle, it should also be applicable to other data sets and prediction tasks. One such task is the prediction of protein functions as specified by the Gene Ontology in combination with protein data from UniProtKB. As an orientation, we can use the DeepGO paper which proposes a solution for this exact task. The goal is to apply our model to the GO / UniProtKB datasets and compare the results to those of DeepGO.

Tasks

1. Minimal dataset implementation: Build a dataset class that extracts proteins and labels from UniProtKB / GO and processes them into a dataset that can be used to train Electra (one-hot encoding of trigrams of amino acids)
2. Model training and evaluation: Evaluate using the same metrics as DeepGO for comparing the models
3. Additonal features and finetuning: Pretraining with additional unlabeled protein data, trained input embeddings, hyperparameters

[aditya0by0]

Protein Preprocessing Statistics

These are the statistics for the proteins that were ignored during preprocessing due to either non-valid amino acids or sequence lengths greater than 1002, as per the guidelines outlined in the paper:

• Number of proteins with non-valid amino acids: 2,672 (0.47% of the dataset)
• Number of proteins with sequence length greater than 1002: 19,004 (3.32% of the dataset)
• Number of proteins with both non-valid amino acids and length greater than 1002: 154
• Total number of ignored proteins (either condition): 21,522 (3.76% of the dataset)
• Original dataset size: 571,864 proteins

The number of ignored proteins is very insignificant in size compared to the whole dataset.

I have attached the CSV file which lists the IDs of the ignored proteins for reference.
proteins_with_issues.csv

[sfluegel05]

Shorten input sequence lengths:

• parameter for maximum length (default: 1002)
• trigrams / n-grams combining several amino acids into one token

[aditya0by0]

Note: The issue will be implemented in 3 PRs:

• Protein function prediction with GO #39
• Protein function prediction with GO - Part 2 #57
• Protein function prediction with GO - Part 3 #64

[sfluegel05]

From #39 (comment):

If 1002 is set as the maximum input sequence length, the updated behavior will truncate any protein sequences longer than 1002 amino acids, selecting only the first 1002. This may result in a partial representation of the protein, as the entire sequence may not be captured.
In contrast, the approach described in the DeepGO paper excludes any protein sequences that exceed the specified length threshold, skipping them entirely rather than truncating.

Excluding long sequences instead of truncating them seems to be the better option. We should implement that

[aditya0by0]

Excluding long sequences instead of truncating them seems to be the better option. We should implement that

I performed a quick analysis by truncating all the protein sequences to different maximum lengths and then checked for duplicates based on the truncated sequences. The results show that as we reduce the maximum sequence length, the number of duplicates increases significantly.

Here are the results for different truncation lengths:

import pandas as pd
df = pd.DataFrame(pd.read_pickle("data/GO_UniProt/GO250_BP/processed/data.pkl"))
df['first_1002'] = df['sequence'].str[:1002]
df['first_700'] = df['sequence'].str[:700]
df['first_500'] = df['sequence'].str[:500]
df['first_200'] = df['sequence'].str[:200]

# Checking for duplicates in the original sequences
df.groupby('sequence').filter(lambda x: len(x) > 1).shape
# (476, 1053)

# Checking for duplicates with truncated sequences
df.groupby('first_1002').filter(lambda x: len(x) > 1).shape
# (480, 1053)

df.groupby('first_700').filter(lambda x: len(x) > 1).shape
# (503, 1053)

df.groupby('first_500').filter(lambda x: len(x) > 1).shape
# (545, 1053)

df.groupby('first_200').filter(lambda x: len(x) > 1).shape
# (1011, 1053)

[sfluegel05]

I still get models that don't train at all, even if reducing the input data length. Two things we should do that (possibly) help with the GO-task:

• find GO classes that should be easy to predict based on the protein structure. In ChEBI, equivalent classes would be chemical entity (everything) or organic molecular entity (everything with carbon). Training only on these classes should be easier
• add pretraining to the pipeline: For chemicals, we use Electra pretraining on PubChem data before applying the model to ChEBI. Similarly, we could use protein data for pretraining here
  • find a data source - does the SwissProt dataset have some proteins that are not annotated with GO labels?