Skip to content

Commit 45e7810

Browse files
authored
Update tokenizer to include tekken implementation (#13601)
Include this meta-pytorch/tokenizers@91140f7
1 parent 8aca197 commit 45e7810

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

extension/llm/tokenizers

0 commit comments

Comments
 (0)