Reproducing DIODE experiments

[theoxo]

Hi there! Thanks for your great work on 4M, super cool stuff.

I'm trying to reproduce some of your results but have been struggling to understand how you formatted the input data. Unfortunately the configs you've pushed to this repo appear pretty sparse. Could you help me understand how to reproduce your DIODE experiments, in particular what preprocessing (if any) of the dataset you carried out and what the generation config was?

Thanks!