execute_mlde.py getting only to 23%

[chelseashuu]

Hello! I have been trying to run execute_mlde but kept running into conflicts between theano, libcublas, pygpu, cuDNN, and tensorflow. The closest I've gotten to a successful run is on GPUs and reached 23% in default training. However, that is the farthest the program will go. Even after 48+ hours the program doesn't run any further and doesn't generate the specified outputs. Is this a common occurrence? Do I need to allocate more GPUs?

Thank you

[brucejwittmann]

@chelseashuu are you using the mlde or mlde2 conda environments? The versions of the packages you mention should be compatible in there.

You certainly shouldn't need more GPUs. All models within MLDE should work just fine on a CPU unless you are working with a truly massive combinatorial space (much more than the 4 sites in the original paper).

If you are working within the mlde or mlde2 environments, can you copy your exact inputs and the console logs? That would help with troubleshooting.

Also, I don't typically check this repo anymore. If you need help, please tag me, otherwise I might take a while to get back to you.

[chelseashuu]

Hi @brucejwittmann thank you for the reply! I had some discrepancies between the tensorflow package in mlde and the GPUs I was using so I was able to resolve the issue!

I did have a follow-up, the fitness scores I generated were all negative, ranging from 0 to -63. Is that right? The paper states that fitness scores should range from 0 - 1 right? Am I missing a crucial calculation or parameter?

[brucejwittmann]

Hi @chelseashuu , glad you got it working! What model is generating those fitness scores? There are no hard bounds on any of the supervised models, so if you're training a model on a dataset with negative fitness values, then you will see predictions of negative fitness values too.

[chelseashuu]

Hi @brucejwittmann! So I just combed through my data and found that the .csv file generated from zero shot prediction contains only negative values. Even the order in which the AA combinations are ranked makes logical sense, the values are just off putting. I used esm1_t6_43M_UR50S in the generate encoding step and esm1b_t33_650M_UR50S in the zero shot prediction step. Does this information allude to any potential errors?

Thank you!

[brucejwittmann]

Hi @chelseashuu, getting only negative values from zero-shot prediction with any of the ESM models is expected. These are not true fitness values--they can be thought of as the log probability that a given amino acid combination would be functional. Because probability will always be between 0 and 1, the log of the probability will always be negative.

You should not need to use any encodings for zero-shot prediction. Zero-shot prediction is really about extracting information from the ESM or other models absent any labeled data. Encodings are used when you've gathered labeled data from the laboratory (perhaps informed by your zero-shot predictions) and are wanting to train a supervised machine learning model.