How much time did it take to run the further pre-training step?

[Vedant-06]

@xuyige Time taken

!python run_pretraining.py
--input_file=./tmp/tf_examples.tfrecord
--output_dir=./tmp/pretraining_output
--do_train=True
--do_eval=True
--bert_config_file=./uncased_L-12_H-768_A-12/bert_config.json
--init_checkpoint=./uncased_L-12_H-768_A-12/bert_model.ckpt
--train_batch_size=32
--max_seq_length=128
--max_predictions_per_seq=20
--num_train_steps=100000
--num_warmup_steps=10000
--learning_rate=5e-5
--use_tpu=False
--save_checkpoints_steps=10000

[xuyige]

We run on a single 1080Ti GPU for about 8-10 hours for 100k steps.