Questions about how to run pretrain on multiple gpus

[ToniChopp]

Hello, I have been trying to reproduce your REFERS code after reading your masterpiece paper. But when I read you codes, I have a question.

• You wrap torch.distributed in https://github.com/funnyzhou/REFERS/blob/master/Pre-train/refers/utils/distributed.py, as I understand it, it is a well-wrapped function that can be operated on multiple gpus on one or even multiple machines. But when I tried to run pretrain_refers.py on 2 gpus, the program crushed down after 100 iterations. So my question is how can I run pretrain on multiple gpus?