InterwebAlchemy
/

PGNTokenizer

Model card Files Files and versions

PGNTokenizer / tokenizer.json

InterwebAlchemy's picture

InterwebAlchemy

Revert back to NFD tokenizer

3341d84 verified about 1 year ago

history contribute delete

263 kB

File too large to display, you can check the raw version instead.