Commit ae7fc2e
committed
Add tp-size and pp-size variations to GPT-J model script
Fixes #671
Added missing tensor parallelism (tp-size) and pipeline parallelism
(pp-size) variations to the get-ml-model-gptj script to match the
implementation in get-ml-model-llama2.
Changes:
- Added tp-size.# and pp-size.# variation definitions
- Set default tp-size.1 and pp-size.1 for pytorch,nvidia variation
- Added MLC_NVIDIA_TP_SIZE and MLC_NVIDIA_PP_SIZE to new_env_keys
This resolves the error "no scripts were found with tags:
get,ml-model,gptj,_nvidia,_fp8,_tp-size.2" when users try to set
MLC_NVIDIA_TP_SIZE environment variable.1 parent cd96a8d commit ae7fc2e
1 file changed
+12
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
| 20 | + | |
19 | 21 | | |
20 | 22 | | |
21 | 23 | | |
| |||
152 | 154 | | |
153 | 155 | | |
154 | 156 | | |
| 157 | + | |
| 158 | + | |
155 | 159 | | |
156 | 160 | | |
157 | 161 | | |
| |||
253 | 257 | | |
254 | 258 | | |
255 | 259 | | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
256 | 268 | | |
257 | 269 | | |
258 | 270 | | |
| |||
0 commit comments