devngho
/

phi3-jamo-ko-tokenizer

Model card Files Files and versions

based on phi-3 tokenizer, expanded 17291 tokens

Following The Optimal Vocabulary Size Predictor, I recommend using this tokenizer with 3-4B model such as phi-3-mini

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Space using devngho/phi3-jamo-ko-tokenizer 1