hf-internal-testing

/

tiny-bert

like 0

Follow

Hugging Face Internal Testing Organization 165

This is a problem that comes up on quite a few models posted here, actually. There's a field in the tokenizer which should represent the maximum length of the model, which in this case is 512 based on the sizes of its internal tensors, and that field is often not set correctly on various models posted to HF.

For example:

https://huggingface.co/google/muril-large-cased/discussions/1

Is this something we can get fixed on the testing model and hopefully elsewhere as well? Thanks!