Accelerate training via training on multiple gpus

[Championchess]

Thanks for your great work! May I wonder whether there is a way to train the model parallelly on multiple gpus so that the training could be speeded up?

[weijiawu]

Sorry, we haven't optimized this part as we only trained on a small amount of data.