Multiple GPU training

[rahimentezari]

Thanks for sharing the code. I was going to use your code to train SimCLR on ImageNet-1K, but could not use multiple gpus on one machine. Can you please let me know how I should use multiple gpus? also how the hyperparameters change with respect to number of gpus?

Secondly, what do you recommend for ImageNet1K hyperparameters, e.g. LR, batch-size, etc?

[mitchellnw]

not an official answer but for the first question of how you can train on multiple gpus you can do torchrun --nproc_per_node=<how-many-gpus> main.py