LangMAP

company

https://cimeister.github.io/blog/unigramlm/

AI & ML interests

Language Adaptive Tokenization

Recent Activity

suchirsalhan updated a model about 1 month ago

MultilingualUnigramLM/ft-langmap-qwen2_5-0_5b-fineweb100M-hun-original-r16k32

suchirsalhan published a model about 1 month ago

MultilingualUnigramLM/ft-langmap-qwen2_5-0_5b-fineweb100M-hun-original-r16k32

suchirsalhan updated a model about 1 month ago

MultilingualUnigramLM/ft-langmap-qwen2_5-0_5b-fineweb100M-hun-langspec-r16k32

View all activity

MultilingualUnigramLM 's datasets 35

MultilingualUnigramLM/FineWeb2-500M-tur_Latn

Viewer • Updated May 18 • 419k • 41

MultilingualUnigramLM/FineWeb2-500M-hun_Latn

Viewer • Updated May 18 • 312k • 42

MultilingualUnigramLM/FineWeb2-500M-fin_Latn

Viewer • Updated May 18 • 387k • 64 • 1

MultilingualUnigramLM/FineWeb2-khm_Khmr-100M

Viewer • Updated May 5 • 1.01M • 9

MultilingualUnigramLM/FineWeb2-tha_Thai-100M

Viewer • Updated May 5 • 710k • 9

MultilingualUnigramLM/FineWeb2-tam_Taml-100M

Viewer • Updated May 5 • 264k • 19

MultilingualUnigramLM/FineWeb2-arb_Arab-100M

Viewer • Updated May 5 • 188k • 10

MultilingualUnigramLM/FineWeb2-ben_Beng-100M

Viewer • Updated May 5 • 254k • 16

MultilingualUnigramLM/FineWeb2-amh_Ethi-100M

Viewer • Updated May 5 • 197k • 21

MultilingualUnigramLM/FineWeb2-yor_Latn-100M

Viewer • Updated May 5 • 80k • 8

MultilingualUnigramLM/FineWeb2-wol_Latn-100M

Viewer • Updated May 5 • 6.62k • 8

MultilingualUnigramLM/FineWeb2-hin_Deva-100M

Viewer • Updated May 5 • 193k • 7

MultilingualUnigramLM/FineWeb2-swh_Latn-100M

Viewer • Updated May 5 • 240k • 7

MultilingualUnigramLM/FineWeb2-deu_Latn-100M

Viewer • Updated May 5 • 234k • 7

MultilingualUnigramLM/FineWeb2-tur_Latn-100M

Viewer • Updated May 3 • 263k • 17

MultilingualUnigramLM/FineWeb2-fin_Latn-100M

Viewer • Updated May 3 • 261k • 11

MultilingualUnigramLM/FineWeb2-hun_Latn-100M

Viewer • Updated May 3 • 198k • 7

MultilingualUnigramLM/LangMap-TheStack-go-100M

Viewer • Updated Apr 13 • 35k • 10

MultilingualUnigramLM/LangMap-TheStack-typescript-100M

Viewer • Updated Apr 13 • 64.2k • 11

MultilingualUnigramLM/LangMap-TheStack-markdown-100M

Viewer • Updated Apr 13 • 56.5k • 11

MultilingualUnigramLM/LangMap-TheStack-css-100M

Viewer • Updated Apr 13 • 18.6k • 7

MultilingualUnigramLM/LangMap-TheStack-html-100M

Viewer • Updated Apr 13 • 20.3k • 16

MultilingualUnigramLM/LangMap-TheStack-php-100M

Viewer • Updated Apr 13 • 78.3k • 10

MultilingualUnigramLM/LangMap-TheStack-csharp-100M

Viewer • Updated Apr 13 • 79.5k • 16 • 1

MultilingualUnigramLM/LangMap-TheStack-cpp-100M

Viewer • Updated Apr 13 • 26.4k • 9

MultilingualUnigramLM/LangMap-TheStack-c-100M

Viewer • Updated Apr 13 • 28.6k • 7

MultilingualUnigramLM/LangMap-TheStack-java-100M

Viewer • Updated Apr 13 • 73k • 12

MultilingualUnigramLM/LangMap-TheStack-javascript-100M

Viewer • Updated Apr 13 • 47.4k • 9

MultilingualUnigramLM/LangMap-TheStack-python-100M

Viewer • Updated Apr 13 • 49.9k • 8 • 1

MultilingualUnigramLM/FineWeb2-100M

Updated Apr 11 • 3