[Test] pt-nightly-hf-mlm-roberta-b-pre-conv-v2-8-1vm without functionalization#868
[Test] pt-nightly-hf-mlm-roberta-b-pre-conv-v2-8-1vm without functionalization#868ManfeiBai wants to merge 3 commits intoGoogleCloudPlatform:masterfrom
Conversation
|
@ManfeiBai You can try one shot tests here: ml-testing-accelerators/tests/README.md Line 36 in 583e71a |
| pip install tensorboardX google-cloud-storage | ||
| echo 'export PATH=~/.local/bin:$PATH' >> ~/.bash_profile | ||
| echo 'export XLA_USE_BF16=1' >> ~/.bash_profile | ||
| echo 'export XLA_DISABLE_FUNCTIONALIZATION=1' >> ~/.bash_profile |
There was a problem hiding this comment.
Can we have a new tpuVm (tpuVmNoFunc) setup that inherits tpuVm and only use that for "hf_lm + v2_8 + roberta_base_pre"?
You can see how pjrt doest it: https://github.com/GoogleCloudPlatform/ml-testing-accelerators/blob/master/tests/pytorch/nightly/resnet50-mp.libsonnet#L188
@will-cromar Feel free to chime in.
There was a problem hiding this comment.
Will you be using it in other tests? If it's just for this file, we can add a mixin here that sets this variable, e.g.
noFunctionalization:: {
tpuSettings+: {
tpuvmExports+: |||
XLA_DISABLE_FUNCTIONALIZATION=1
|||
}
}
The correct place to set environment variables for these tests is tpuVmExports+. The use of .bash_profile is probably just historical baggage.
If it's shared by other tests, feel free to add this field to common instead.
Description
Please include a summary of relevant context/issue and your changes.
Tests
Please describe the tests that you ran on TPUs to verify changes.
Instruction and/or command lines to reproduce your tests: ...
List links for your tests (use go/shortn-gen for any internal link): ...
Checklist
Before submitting this PR, please make sure (put X in square brackets):