Bug in loading state_dict for Dinov2Embeddings
I get this bug running any of the Pytorch code in your README.
model = AutoModel.from_pretrained("TrungTran/faceage_ClientScan", trust_remote_code=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\models\auto\auto_factory.py", line 597, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 277, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 5048, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 5468, in _load_pretrained_model
_error_msgs, disk_offload_index = load_shard_file(args)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 843, in load_shard_file
disk_offload_index = _load_state_dict_into_meta_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\torch\utils_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 770, in _load_state_dict_into_meta_model
_load_parameter_into_model(model, param_name, param.to(param_device))
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\transformers\modeling_utils.py", line 667, in _load_parameter_into_model
module.load_state_dict({param_type: tensor}, strict=False, assign=True)
File "C:\Users\Tony\miniforge3\envs\ryzen-ai-1.7.1\Lib\site-packages\torch\nn\modules\module.py", line 2639, in load_state_dict
raise RuntimeError(
RuntimeError: Error(s) in loading state_dict for Dinov2Embeddings:
size mismatch for mask_token: copying a param with shape torch.Size([1, 1, 1024]) from checkpoint, the shape in current model is torch.Size([1, 1024]).
No problem using the ONNX model, though. It's amazing.
Hi @Gatecrasher777 ,
So sorry for the bug. I guess the problem is I'm using the backbone Dinov3, and the code auto-casts to Dinov2. That 's why the mismatch happened.
Could you please ask for the permission on : facebook/dinov3-vitl16-pretrain-lvd1689m? Then, please upgrade the library for transformers.pip install -U "transformers>=4.56.0" "torch>=2.1" "safetensors" "pillow" "numpy"
I'm using transformer 5.5.0.
Btw, the performance for Onnx is the same to the pytorch code but Pytorch support GPUs.
Please let me know if the problem is fixed, or I will find other solution.