This repository is a place for me to experiment with pretraining of small (<= ~3B) language models.
This repository assumes you have access to a modal account.
`modal run -m scripts.01_create_data_volume::main`SJCaldwell/nanoPT
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|