splade_morpheme / tokenizer_config.json
HeyDunaX's picture
upload tokenizer_config.json
a5d98a6 verified
{
"tokenizer_class": "MorphemeTokenizer",
"model_type": "splade",
"vocab_size": 57030,
"max_length": 128,
"do_lower_case": false,
"seg_temp_train": 1.0,
"seg_temp_eval": 0.0,
"min_freq_word": 2,
"min_freq_morph": 3
}