LnL-AI
/

dbrx-base-tokenizer

Transformers.js

Model card Files Files and versions

config.json embedding size of "vocab_size": 100352 does not match 100277

#6

by treehugg3 - opened Aug 12, 2025

•

edited Aug 12, 2025

Did you ever find a fix for this? I cannot use it with llama.cpp because of this mistake.

Relevant: https://huggingface.co/databricks/dbrx-base/discussions/18

I fixed it here by extending the vocabulary: https://huggingface.co/treehugg3/dbrx-base-tokenizer-llamacpp

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment