Initial commit for model card
#1
by Lexiyutou - opened
README.md
CHANGED
|
@@ -1,3 +1,95 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- computer_vision
|
| 4 |
+
- animal_pose_and_shape_estimation
|
| 5 |
+
- DeepLabCut
|
| 6 |
+
pipeline_tag: image-to-3d
|
| 7 |
+
---
|
| 8 |
+
# MODEL CARD:
|
| 9 |
+
|
| 10 |
+
## Model Details
|
| 11 |
+
|
| 12 |
+
• PRIMA model(s) developed by the [M.W.Mathis Lab](http://www.mackenziemathislab.org/) in 2026, trained to predict quadruped shape and pose from images.
|
| 13 |
+
Please see **paper link** for details.
|
| 14 |
+
|
| 15 |
+
• There are two main models:
|
| 16 |
+
- `s1ckpt.ckpt` is the stage-1 model trained with Animal3D, CtrlAni3D, and Quadruped2D datasets.
|
| 17 |
+
- `s3ckpt.ckpt` is the stage-3 model trained with Animal3D, CtrlAni3D, and Quadruped3D datasets.
|
| 18 |
+
|
| 19 |
+
```python
|
| 20 |
+
from pathlib import Path
|
| 21 |
+
from dlclibrary import download_huggingface_model
|
| 22 |
+
# Creates a folder and downloads the model to it
|
| 23 |
+
model_dir = Path("./prima_animal_model_pytorch")
|
| 24 |
+
model_dir.mkdir()
|
| 25 |
+
download_huggingface_model("prima_animal_model_pytorch", model_dir)
|
| 26 |
+
```
|
| 27 |
+
## Intended Use
|
| 28 |
+
• Intended to be used for shape and pose estimation of quadruped images taken from a single view.
|
| 29 |
+
|
| 30 |
+
• Intended for academic and research professionals working in fields related to animal behavior, such as neuroscience
|
| 31 |
+
and ecology.
|
| 32 |
+
|
| 33 |
+
• Not suitable as a zero-shot model for applications that require high shape and pose precision, but can be further optimized with 2D keypoint
|
| 34 |
+
annotations or from SuperAnimal to improve accuracy. Also, it is not suitable for videos that look dramatically different from those
|
| 35 |
+
we show in the paper.
|
| 36 |
+
|
| 37 |
+
## Metrics
|
| 38 |
+
• PA-MPJPE (Procrustes-aligned mean per-joint position error), computed over 3D joints.
|
| 39 |
+
|
| 40 |
+
• PA-MPVPE (Procrustes-aligned mean per-vertex position error), computed over the SMAL mesh vertices.
|
| 41 |
+
|
| 42 |
+
• PCK (Percentage of Correct Keypoints) measures the proportion of predicted keypoints within a specified threshold of the ground-truth keypoints.
|
| 43 |
+
|
| 44 |
+
• AUC (Area Under the Curve), computed by integrating the PCK values as the threshold varies from 0 to 1.
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
## Evaluation Data
|
| 48 |
+
• In the paper we benchmark on Animal3d, CtrlAni3D, Quadruped2D, and AnimalKingdom.
|
| 49 |
+
|
| 50 |
+
## Training Data:
|
| 51 |
+
It consists of being trained together on the following datasets:
|
| 52 |
+
- **Animal3D** see full details at (1).
|
| 53 |
+
- **CtrlAni3D** See full details at (2).
|
| 54 |
+
- **Quadruped2D** See full details at (3).
|
| 55 |
+
- **Quadruped3D** See full details at **paper link**.
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
## Ethical Considerations
|
| 59 |
+
• No experimental data were collected for this model; all datasets used are cited.
|
| 60 |
+
|
| 61 |
+
## License
|
| 62 |
+
Modified MIT.
|
| 63 |
+
|
| 64 |
+
Copyright 2026 by Mackenzie Mathis, Xiaohang Yu, and contributors.
|
| 65 |
+
|
| 66 |
+
Permission is hereby granted to you (hereafter "LICENSEE") a fully-paid, non-exclusive,
|
| 67 |
+
and non-transferable license for academic, non-commercial purposes only (hereafter “LICENSE”)
|
| 68 |
+
to use the "MODEL" weights (hereafter "MODEL"), subject to the following conditions:
|
| 69 |
+
|
| 70 |
+
The above copyright notice and this permission notice shall be included in all copies or substantial
|
| 71 |
+
portions of the Software:
|
| 72 |
+
|
| 73 |
+
This software may not be used to harm any animal deliberately.
|
| 74 |
+
|
| 75 |
+
LICENSEE acknowledges that the MODEL is a research tool.
|
| 76 |
+
THE MODEL IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
|
| 77 |
+
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
| 78 |
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
| 79 |
+
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL
|
| 80 |
+
OR THE USE OR OTHER DEALINGS IN THE MODEL.
|
| 81 |
+
|
| 82 |
+
If this license is not appropriate for your application, please contact Prof. Mackenzie W. Mathis
|
| 83 |
+
(mackenzie@post.harvard.edu) and/or the TTO office at EPFL (tto@epfl.ch) for a commercial use license.
|
| 84 |
+
|
| 85 |
+
Please cite **paper link** if you use this model in your work.
|
| 86 |
+
|
| 87 |
+
## References
|
| 88 |
+
1. Xu, J., Zhang, Y., Peng, J., Ma, W., Jesslen, A., Ji, P., Hu, Q., Zhang, J., Liu, Q.,
|
| 89 |
+
Wang, J., et al.: Animal3d: A comprehensive dataset of 3d animal pose and shape.
|
| 90 |
+
In: ICCV. pp. 9099–9109 (2023)
|
| 91 |
+
2. Lyu, J., Zhu, T., Gu, Y., Lin, L., Cheng, P., Liu, Y., Tang, X., An, L.: Animer:
|
| 92 |
+
Animal pose and shape estimation using a family-aware transformer. In: CVPR. pp.
|
| 93 |
+
17486–17496 (2025)
|
| 94 |
+
3. Ye, S., Filippova, A., Lauer, J., Schneider, S., Vidal, M., Qiu, T., Mathis, A.,
|
| 95 |
+
Mathis, M.W.: Superanimal pretrained pose estimation models for behavioral analysis. Nature communications 15(1), 5165 (2024)
|