Skip to content

Commit 974fadf

Browse files
authored
Update README.md
1 parent ae32cd1 commit 974fadf

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,11 @@ Install this library as a local editable installation. Run the following command
1616

1717
To run the default pipeline from the command line, use the following command:
1818

19-
`python -m delphi meta-llama/Meta-Llama-3-8B EleutherAI/sae-llama-3-8b-32x --explainer_model 'hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4' --dataset_repo 'EleutherAI/fineweb-edu-dedup-10b' --dataset_split 'train[:1%]' --n_tokens 10_000_000 --max_latents 100 --hookpoints layers.5 --scorers detection --filter_bos --name llama-3-8B`
19+
`python -m delphi meta-llama/Meta-Llama-3-8B EleutherAI/sae-llama-3-8b-32x --n_tokens 10_000_000 --max_latents 100 --hookpoints layers.5 --scorers detection --filter_bos --name llama-3-8B`
2020

2121
This command will:
22-
1. Cache activations for the first 10 million tokens of the dataset.
23-
2. Generate explanations for the first 100 features of layer 5 using the specified explainer model.
22+
1. Cache activations for the first 10 million tokens of the default dataset, `EleutherAI/SmolLM2-135M-10B`.
23+
2. Generate explanations for the first 100 features of layer 5 using the default explainer model, `hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4`.
2424
3. Score the explanations using the detection scorer.
2525
4. Log summary metrics including per-scorer F1 scores and confusion matrices, and produce histograms of the scorer classification accuracies.
2626

@@ -36,7 +36,7 @@ The first step to generate explanations is to cache sparse model activations. To
3636
from sparsify.data import chunk_and_tokenize
3737
from delphi.latents import LatentCache
3838

39-
data = load_dataset("EleutherAI/rpj-v2-sample", split="train[:1%]")
39+
data = load_dataset("EleutherAI/SmolLM2-135M-10B", split="train[:1%]")
4040
tokens = chunk_and_tokenize(data, tokenizer, max_seq_len=256, text_key="raw_content")["input_ids"]
4141

4242
cache = LatentCache(

0 commit comments

Comments
 (0)