| --- |
| tags: |
| - computer_vision |
| - animal_pose_and_shape_estimation |
| - DeepLabCut |
| pipeline_tag: image-to-3d |
| --- |
| # MODEL CARD: |
|
|
| ## Model Details |
|
|
| • PRIMA model(s) developed by the [M.W.Mathis Lab](http://www.mackenziemathislab.org/) in 2026, trained to predict quadruped shape and pose from images. |
| Please see **paper link** for details. |
|
|
| • There are two main models: |
| - `s1ckpt.ckpt` is the stage-1 model trained with Animal3D, CtrlAni3D, and Quadruped2D datasets. |
| - `s3ckpt.ckpt` is the stage-3 model trained with Animal3D, CtrlAni3D, and Quadruped3D datasets. |
|
|
| ```python |
| from pathlib import Path |
| from huggingface_hub import hf_hub_download |
| |
| repo_id = "MLAdaptiveIntelligence/PRIMA" |
| |
| model_dir = Path("./prima_model") |
| model_dir.mkdir(parents=True, exist_ok=True) |
| |
| # download stage-1 checkpoint |
| s1_path = hf_hub_download( |
| repo_id=repo_id, |
| filename="s1ckpt.ckpt", |
| local_dir=model_dir |
| ) |
| |
| # donwload stage-3 checkpoint |
| s3_path = hf_hub_download( |
| repo_id=repo_id, |
| filename="s3ckpt.ckpt", |
| local_dir=model_dir |
| ) |
| ``` |
| ## Intended Use |
| • Intended to be used for shape and pose estimation of quadruped images taken from a single view. |
|
|
| • Intended for academic and research professionals working in fields related to animal behavior, such as neuroscience |
| and ecology. |
|
|
| • Not suitable as a zero-shot model for applications that require high shape and pose precision, but can be further optimized with 2D keypoint |
| annotations or from SuperAnimal to improve accuracy. Also, it is not suitable for videos that look dramatically different from those |
| we show in the paper. |
|
|
| ## Metrics |
| • PA-MPJPE (Procrustes-aligned mean per-joint position error), computed over 3D joints. |
|
|
| • PA-MPVPE (Procrustes-aligned mean per-vertex position error), computed over the SMAL mesh vertices. |
|
|
| • PCK (Percentage of Correct Keypoints) measures the proportion of predicted keypoints within a specified threshold of the ground-truth keypoints. |
|
|
| • AUC (Area Under the Curve), computed by integrating the PCK values as the threshold varies from 0 to 1. |
|
|
|
|
| ## Evaluation Data |
| • In the paper we benchmark on Animal3d, CtrlAni3D, Quadruped2D, and AnimalKingdom. |
|
|
| ## Training Data: |
| It consists of being trained together on the following datasets: |
| - **Animal3D** see full details at (1). |
| - **CtrlAni3D** See full details at (2). |
| - **Quadruped2D** See full details at (3). |
| - **Quadruped3D** See full details at **paper link**. |
|
|
|
|
| ## Ethical Considerations |
| • No experimental data were collected for this model; all datasets used are cited. |
|
|
| ## License |
| Modified MIT. |
|
|
| Copyright 2026 by Mackenzie Mathis, Xiaohang Yu, and contributors. |
|
|
| Permission is hereby granted to you (hereafter "LICENSEE") a fully-paid, non-exclusive, |
| and non-transferable license for academic, non-commercial purposes only (hereafter “LICENSE”) |
| to use the "MODEL" weights (hereafter "MODEL"), subject to the following conditions: |
|
|
| The above copyright notice and this permission notice shall be included in all copies or substantial |
| portions of the Software: |
|
|
| This software may not be used to harm any animal deliberately. |
|
|
| LICENSEE acknowledges that the MODEL is a research tool. |
| THE MODEL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING |
| BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. |
| IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, |
| WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL |
| OR THE USE OR OTHER DEALINGS IN THE MODEL. |
|
|
| If this license is not appropriate for your application, please contact Prof. Mackenzie W. Mathis |
| (mackenzie@post.harvard.edu) and/or the TTO office at EPFL (tto@epfl.ch) for a commercial use license. |
|
|
| Please cite **paper link** if you use this model in your work. |
|
|
| ## References |
| 1. Xu, J., Zhang, Y., Peng, J., Ma, W., Jesslen, A., Ji, P., Hu, Q., Zhang, J., Liu, Q., |
| Wang, J., et al.: Animal3d: A comprehensive dataset of 3d animal pose and shape. |
| In: ICCV. pp. 9099–9109 (2023) |
| 2. Lyu, J., Zhu, T., Gu, Y., Lin, L., Cheng, P., Liu, Y., Tang, X., An, L.: Animer: |
| Animal pose and shape estimation using a family-aware transformer. In: CVPR. pp. |
| 17486–17496 (2025) |
| 3. Ye, S., Filippova, A., Lauer, J., Schneider, S., Vidal, M., Qiu, T., Mathis, A., |
| Mathis, M.W.: Superanimal pretrained pose estimation models for behavioral analysis. Nature communications 15(1), 5165 (2024) |
