Spaces:

Duplicated from TWO/sutra-tokenizer-comparison

mikesapi
/

sutra-tokenizer-comparison

Sleeping

App Files Files Community

sutra-tokenizer-comparison / data

42.2 kB

2 contributors

History: 2 commits

Michael Sapienza

adding more examples

5be3066 almost 2 years ago

multilingualphrases01.txt

24.4 kB

adding sutra tokenizer inspector app almost 2 years ago
multilingualphrases02.txt

11.3 kB

adding sutra tokenizer inspector app almost 2 years ago
multilingualphrases03.txt

6.51 kB

adding more examples almost 2 years ago