You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/getting-started/download_models.md
+30-12Lines changed: 30 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,30 +6,46 @@ logo: images/ilab_dog.png
6
6
7
7
# 📥 Download the model
8
8
9
-
- Run the `ilab model download` command.
9
+
1) Run the ilab model download command to download a compact pre-trained version of the `granite-7b-lab-GGUF`, `merlinite-7b-lab-GGUF`, and `Mistral-7B-Instruct-v0.2-GGUF` models (~4.4G each) from HuggingFace.
10
10
11
11
```shell
12
12
ilab model download
13
13
```
14
14
15
15
`ilab model download` downloads a compact pre-trained version of the [model](https://huggingface.co/instructlab/) (~4.4G) from HuggingFace:
TheBloke/Mistral-7B-Instruct-v0.2-GGUF requires a HF Token to be set.
31
+
Please use '--hf-token' or 'export HF_TOKEN' to download all necessary models.
32
+
```
33
+
34
+
a) You may be prompted to use your Hugging Face token to download the `Mistral-7B-Instruct-v0.2-GGUF` model.
35
+
17
36
```shell
18
-
(venv) $ ilab model download
19
-
Downloading model from Hugging Face: instructlab/merlinite-7b-lab-GGUF@main to /Users/USERNAME/Library/Caches/instructlab/models...
20
-
...
21
-
INFO 2024-08-01 15:05:48,464 huggingface_hub.file_download:1893: Download complete. Moving file to /Users/USERNAME/Library/Caches/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf
37
+
ilab model download --hf-token <your-huggingface-token>
22
38
```
23
39
24
40
!!! note
25
41
⏳ This command can take few minutes or immediately depending on your internet connection or model is cached. If you have issues connecting to Hugging Face, refer to the [Hugging Face discussion forum](https://discuss.huggingface.co/) for more details.
26
42
27
43
## Downloading an entire Hugging Face repository (Safetensors Model)
28
44
29
-
- Specify repository, and a Hugging Face token if necessary. For example:
45
+
1) Specify repository, and a Hugging Face token if necessary. For example:
30
46
31
47
```shell
32
-
HF_TOKEN=<YOUR HUGGINGFACE TOKEN GOES HERE>ilab model download --repository=instructlab/granite-7b-lab
48
+
ilab model download --repositoryinstructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf --hf-token <your-huggingface-token>
33
49
```
34
50
35
51
These types of models are useful for GPU-enabled systems or anyone looking to serve a model using vLLM. InstructLab provides Safetensor versions of our Granite models on HuggingFace.
Copy file name to clipboardExpand all lines: docs/getting-started/initilize_ilab.md
+64-88Lines changed: 64 additions & 88 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,112 +6,88 @@ logo: images/ilab_dog.png
6
6
7
7
# 🏗️ Initialize `ilab`
8
8
9
-
### 🏗️ Initialize `ilab`
9
+
1)Initialize `ilab` by running the following command:
10
10
11
-
1. Initialize `ilab` by running the following command:
12
-
13
-
```shell
14
-
ilab config init
15
-
```
11
+
```shell
12
+
ilab config init
13
+
```
16
14
17
-
2. When prompted, clone the `https://github.com/instructlab/taxonomy.git` repository into the current directory by typing **enter**
15
+
2) When prompted, clone the `https://github.com/instructlab/taxonomy.git` repository into the current directory by typing **enter**
18
16
19
17
**Optional**: If you want to point to an existing local clone of the `taxonomy` repository, you can pass the path interactively or alternatively with the `--taxonomy-path` flag.
20
18
21
19
`ilab` will use the default configuration file unless otherwise specified. You can override this behavior with the `--config` parameter for any `ilab` command.
22
20
23
-
3. When prompted, provide the path to your default model. Otherwise, the default of a quantized [Merlinite](https://huggingface.co/instructlab/merlinite-7b-lab-GGUF) model is used.
Please provide the following values to initiate the environment [press Enter for defaults]:
34
-
Path to taxonomy repo [/Users/kellybrown/.local/share/instructlab/taxonomy]:
35
-
Path to your model [/Users/kellybrown/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf]:
36
-
```
37
-
38
-
You can download this model with `ilab model download` command as well.
39
-
40
-
4. The InstructLab CLI auto-detects your hardware and select the exact system profile that matches your machine. System profiles populate the `config.yaml` file with the proper parameter values based on your detected GPU types and avaiible vRAM.
We have detected the AMD CPU profile as an exact match for your system.
50
-
51
-
--------------------------------------------
52
-
Initialization completed successfully!
53
-
You're ready to start using `ilab`. Enjoy!
54
-
--------------------------------------------
55
-
```
56
-
57
-
5. If there is not an exact match for your system, you can manually select a system profile when prompted. There are various flags you can utilize with individual `ilab` commands that allow you to utilize your GPU if applicable.
58
-
59
-
*Example output of selecting a system profile*
60
-
61
-
```shell
62
-
Please choose a system profile to use.
63
-
System profiles apply to all parts of the config file and set hardware specific defaults for each command.
64
-
First, please select the hardware vendor your system falls into
65
-
[1] APPLE
66
-
[2] INTEL
67
-
[3] AMD
68
-
[4] NVIDIA
69
-
Enter the number of your choice [0]: 1
70
-
You selected: APPLE
71
-
Next, please select the specific hardware configuration that most closely matches your system.
72
-
[0] No system profile
73
-
[1] APPLE M1 ULTRA
74
-
[2] APPLE M1 MAX
75
-
[3] APPLE M2 MAX
76
-
[4] APPLE M2 ULTRA
77
-
[5] APPLE M2 PRO
78
-
[6] APPLE M2
79
-
[7] APPLE M3 MAX
80
-
[8] APPLE M3 PRO
81
-
[9] APPLE M3
82
-
Enter the number of your choice [hit enter for hardware defaults] [0]: 8
83
-
You selected: /Users/kellybrown/.local/share/instructlab/internal/system_profiles/apple/m3/m3_pro.yaml
84
-
85
-
--------------------------------------------
86
-
Initialization completed successfully!
87
-
You're ready to start using `ilab`. Enjoy!
88
-
--------------------------------------------
89
-
```
21
+
3) When prompted, provide the path to your default model. Otherwise, the default of a quantized [Merlinite](https://huggingface.co/instructlab/merlinite-7b-lab-GGUF) model is used.
90
22
91
-
The GPU profiles are listed by GPU type and number of GPUs present. If you happen to have a GPU configuration with a similar amount of vRAM as any of the above profiles, feel free to try them out!
Please provide the following values to initiate the environment [press Enter for defaults]:
31
+
Path to taxonomy repo [/Users/<user>/.local/share/instructlab/taxonomy]:
32
+
Path to your model [/Users/<user>/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf]:
33
+
```
92
34
93
-
### `ilab` directory layout after initializing your system
35
+
You can download this model with `ilab model download` command as well.
94
36
95
-
### Mac directory
37
+
4) The InstructLab CLI auto-detects your hardware and select the exact system profile that matches your machine. System profiles populate the `config.yaml` file with the proper parameter values based on your detected GPU types and avaiible vRAM.
96
38
97
-
After running `ilab config init` your directories will look like the following on a Mac system:
1)`/Users/USERNAME/Library/Caches/instructlab/models/`: Contains all downloaded large language models, including the saved output of ones you generate with ilab.
46
+
We have detected the AMD CPU profile as an exact match for your system.
107
47
108
-
2)`~/Library/Application\ Support/instructlab/datasets/`: Contains data output from the SDG phase, built on modifications to the taxonomy repository.
48
+
--------------------------------------------
49
+
Initialization completed successfully!
50
+
You're ready to start using `ilab`. Enjoy!
51
+
--------------------------------------------
52
+
```
109
53
110
-
3)`~/Library/Application\ Support/instructlab/taxonomy/`: Contains the skill and knowledge data.
54
+
5) If there is not an exact match for your system, you can manually select a system profile when prompted. There are various flags you can utilize with individual `ilab` commands that allow you to utilize your GPU if applicable.
111
55
112
-
4)`~/Users/USERNAME/Library/Caches/instructlab/checkpoints/`: Contains the output of the training process
56
+
*Example output of selecting a system profile*
113
57
114
-
### Linux directory
58
+
```shell
59
+
Please choose a system profile to use.
60
+
System profiles apply to all parts of the config file and set hardware specific defaults for each command.
61
+
First, please select the hardware vendor your system falls into
62
+
[1] APPLE
63
+
[2] INTEL
64
+
[3] AMD
65
+
[4] NVIDIA
66
+
Enter the number of your choice [0]: 1
67
+
You selected: APPLE
68
+
Next, please select the specific hardware configuration that most closely matches your system.
69
+
[0] No system profile
70
+
[1] APPLE M1 ULTRA
71
+
[2] APPLE M1 MAX
72
+
[3] APPLE M2 MAX
73
+
[4] APPLE M2 ULTRA
74
+
[5] APPLE M2 PRO
75
+
[6] APPLE M2
76
+
[7] APPLE M3 MAX
77
+
[8] APPLE M3 PRO
78
+
[9] APPLE M3
79
+
Enter the number of your choice [hit enter for hardware defaults] [0]: 8
80
+
You selected: /Users/kellybrown/.local/share/instructlab/internal/system_profiles/apple/m3/m3_pro.yaml
81
+
82
+
--------------------------------------------
83
+
Initialization completed successfully!
84
+
You're ready to start using `ilab`. Enjoy!
85
+
--------------------------------------------
86
+
```
87
+
88
+
The GPU profiles are listed by GPU type and number of GPUs present. If you happen to have a GPU configuration with a similar amount of vRAM as any of the above profiles, feel free to try them out!
89
+
90
+
### `ilab` directory layout after initializing your system
115
91
116
92
After running `ilab config init` your directories will look like the following on a Linux system:
-v, --verbose Enable debug logging (repeat for even more verbosity)
85
-
--version Show the version and exit.
86
-
--help Show this message and exit.
87
-
88
-
Commands:
89
-
config Command Group for Interacting with the Config of InstructLab.
90
-
data Command Group for Interacting with the Data generated by...
91
-
model Command Group for Interacting with the Models in InstructLab.
92
-
system Command group for all system-related command calls
93
-
taxonomy Command Group for Interacting with the Taxonomy of InstructLab.
94
-
95
-
Aliases:
96
-
chat model chat
97
-
generate data generate
98
-
serve model serve
99
-
train model train
100
-
```
84
+
-v, --verbose Enable debug logging (repeat for even more verbosity)
85
+
--version Show the version and exit.
86
+
--help Show this message and exit.
87
+
88
+
Commands:
89
+
config Command Group for Interacting with the Config of InstructLab.
90
+
data Command Group for Interacting with the Data generated by...
91
+
model Command Group for Interacting with the Models in InstructLab.
92
+
system Command group for all system-related command calls
93
+
taxonomy Command Group for Interacting with the Taxonomy of InstructLab.
94
+
95
+
Aliases:
96
+
chat model chat
97
+
generate data generate
98
+
serve model serve
99
+
train model train
100
+
```
101
101
102
102
!!! important
103
103
Every `ilab` command needs to be run from within your Python virtual environment. You can enter the Python environment by running the `source venv/bin/activate` command.
0 commit comments