codefuse-ai

/

F2LLM-v2-80M

like 6

Follow

CodeFuse AI 213

First, let me thank you for this awesome family of embedding models. I'm finding them really useful in my work.

I'm running into an issue when loading some of your v2 models on transformers 5.3.0 (and sentence-transformers 5.3.0).

Currently, all three models I've tested (80M, 160M, and 330M) are throwing this error on load:

Python 3.12.3 (main, Mar  3 2026, 12:15:18) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sentence_transformers import SentenceTransformer
>>> model = SentenceTransformer("codefuse-ai/F2LLM-v2-80M", model_kwargs={"torch_dtype": "bfloat16"})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../sentence_transformers/SentenceTransformer.py", line 327, in __init__
    modules, self.module_kwargs = self._load_sbert_model(
                                  ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../sentence_transformers/SentenceTransformer.py", line 2323, in _load_sbert_model
    module = module_class.load(
             ^^^^^^^^^^^^^^^^^^
  File ".../sentence_transformers/models/Transformer.py", line 436, in load
    return cls(model_name_or_path=model_name_or_path, **init_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../sentence_transformers/models/Transformer.py", line 120, in __init__
    config, is_peft_model = self._load_config(model_name_or_path, cache_dir, backend, config_args)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../sentence_transformers/models/Transformer.py", line 195, in _load_config
    return AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir), False
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../transformers/models/auto/configuration_auto.py", line 1469, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../transformers/configuration_utils.py", line 716, in from_dict
    config = cls(**config_dict)
             ^^^^^^^^^^^^^^^^^^
  File ".../transformers/models/qwen3/configuration_qwen3.py", line 185, in __init__
    layer_type_validation(self.layer_types, self.num_hidden_layers)
  File ".../transformers/configuration_utils.py", line 1250, in layer_type_validation
    raise ValueError(
ValueError: `num_hidden_layers` (8) must be equal to the number of layer types (28)

It seems like each model's config.json has num_hidden_layers set for the distilled sizes (8, 9, and 16). However, the layer_types list still has 28 entries. If I'm understanding the problem correctly it should be a quick config update. I think the layer_types array in each config.json needs to be truncated so it matches the num_hidden_layers:

80M: layer_types[:8]
160M: layer_types[:9]
330M: layer_types[:16]

In the meantime, if anyone else is experiencing this, you can get around this by patching the config before loading the model. Here’s my current workaround:

import json, torch
from transformers import AutoModel, AutoTokenizer, Qwen3Config
from huggingface_hub import hf_hub_download

model_id = "codefuse-ai/F2LLM-v2-80M"

# Pull and patch the config
cfg = json.load(open(hf_hub_download(model_id, 'config.json')))
cfg['layer_types'] = cfg['layer_types'][:cfg['num_hidden_layers']]
config = Qwen3Config(**cfg)

# Load the model
model = AutoModel.from_pretrained(model_id, config=config, torch_dtype=torch.bfloat16)