chrishayuk
/

v11-tokenizer

language-modeling

Model card Files Files and versions

v11-tokenizer / tokenizer.json

chrishayuk's picture

v11: byte-safety fix (0 UNK, 0 round-trip mismatches on the 32-file gate corpus)

7e96a85 verified about 20 hours ago

History Contribute Delete

1.57 MB

File too large to display, you can check the raw version instead.