kgrabko commited on
Commit
6fd8d19
·
verified ·
1 Parent(s): 1f52285

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -12,7 +12,7 @@ license_link: LICENSE
12
  eos_token_id: 128001
13
  }
14
  - Compatible with Llama 3 • Optimized for Code . Use resize function without adaptation, see examples below
15
- - It need 100k example to fully adapt routing features for RAG Routing model . So check llama modification tokenizer rules
16
  - A Llama 3-based tokenizer enhanced with FIM markers from Microsoft datasets (
17
  <|fim_prefix|>,
18
  <|fim_middle|>,
 
12
  eos_token_id: 128001
13
  }
14
  - Compatible with Llama 3 • Optimized for Code . Use resize function without adaptation, see examples below
15
+ - It needs 100k example to fully adapt routing features for RAG Routing model . So check llama modification tokenizer rules
16
  - A Llama 3-based tokenizer enhanced with FIM markers from Microsoft datasets (
17
  <|fim_prefix|>,
18
  <|fim_middle|>,