flexitok
/

parity-aware-bpe-devset

Model card Files Files and versions

parity-aware-bpe-devset

Commit History

added bos and eos to the config

a067e5d
verified

gsaltintas commited on Aug 21

Upload tokenizer.json with huggingface_hub

5865c46
verified

gsaltintas commited on Aug 21

Rename tokenizer.json to tokenizer_old.json

7ddeefa
verified

gsaltintas commited on Aug 21

Upload README.md with huggingface_hub

ab58430
verified

gsaltintas commited on Aug 9

Upload tokenizer

dd6a333
verified

gsaltintas commited on Aug 9

initial commit

d303316
verified

gsaltintas commited on Aug 9