CMSManhattan
/

LlamaRoboticsTokenizer

Mixture of Experts

Model card Files Files and versions

kgrabko commited on 24 days ago

Commit

fea9e79

·

verified ·

1 Parent(s): 887d840

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -19,7 +19,7 @@ tags:
 - Llama Pro Tokenizer . Fully compatible with Llama3 models.
 - Size {
  Vocab size: 128365
- pad_token_id: None
  eos_token_id: 128001
  }
 - Compatible with Llama 3 • Optimized for Code . Use resize function without adaptation, see examples below
@@ -215,5 +215,5 @@ print("eos_token_id:", tok.eos_token_id)
 '
 Vocab size: 128365
-pad_token_id: None
 eos_token_id: 128001

 - Llama Pro Tokenizer . Fully compatible with Llama3 models.
 - Size {
  Vocab size: 128365
+ pad_token_id: 128001
  eos_token_id: 128001
  }
 - Compatible with Llama 3 • Optimized for Code . Use resize function without adaptation, see examples below
 '
 Vocab size: 128365
+pad_token_id: 128001
 eos_token_id: 128001