ZeyuLing/Motius-MotionGPT-HumanML3D

MotionGPT Model Card

Motion-language generation with discrete motion tokens, packaged as a Motius pipeline.

Paper | Project Page | Original GitHub | Motius Checkpoint

MotionGPT is the motion-language baseline from MotionGPT: Human Motion as a Foreign Language (Jiang et al., NeurIPS 2023). This Motius release packages the HumanML3D motion tokenizer, FLAN-T5-base-style language model with motion tokens, HumanML3D statistics, and task-facing text-to-motion / motion-to-text pipeline methods without requiring the original checkout.

Preview

HumanML3D Sample	Input Text	SMPL Preview
001840	someone executes a roundhouse kick with their left foot.
004545	a person jumping while raising both hands and moving apart legs.
006944	a person moves their right hand left, right, up, and down.

512px / 30fps GIF previews rendered from released HumanML3D test outputs.

Release Snapshot

Item	Value
Method	MotionGPT, language modeling over text and motion tokens
Tasks	T2M, M2T
Venue	NeurIPS 2023
Motion representation	HumanML3D-263, 20 fps
Language backbone	FLAN-T5-base-style encoder-decoder with motion tokens
Motion tokenizer	VQ-VAE, 512-code codebook
Checkpoint	ZeyuLing/Motius-MotionGPT-HumanML3D
Pipeline	motius.pipelines.motiongpt.MotionGPTPipeline

The checkpoint artifact contains motiongpt_s3_h3d.tar, assets/meta/mean.npy, assets/meta/std.npy, deps/flan-t5-base/, and model_index.json.

Usage

from motius.pipelines.motiongpt import MotionGPTPipeline

pipe = MotionGPTPipeline.from_pretrained(
    "ZeyuLing/Motius-MotionGPT-HumanML3D",
    bundle_kwargs={"local_files_only": False},
    device="cuda",
)

motions = pipe.infer_t2m(
    ["a person walks forward then sits down"],
    [120],
)

motions is a list of NumPy arrays. Each array has shape (T, 263) and is denormalized to HumanML3D physical scale. The same pipeline also exposes infer_m2t for captioning denormalized HumanML3D-263 motions.

caption = pipe.infer_m2t(
    [motions[0]],
    lengths=[len(motions[0])],
)[0]

Evaluation Results

Protocol: HumanML3D Official uses the selected-caption HumanML3D test protocol. MotionStreamer Evaluator and Motius Joint-Position Evaluator are computed after converting outputs through the shared SMPL-22 evaluation bridge. For FID and MM-Dist, lower is better.

Evaluator	Variant	Samples	R@1	R@2	R@3	FID	MM-Dist	Diversity	Status
HumanML3D Official	Default	3,962	0.434	0.600	0.686	0.156	3.920	9.747	Measured
MotionStreamer Evaluator	Default	4,042	0.494	0.635	0.694	23.681	19.678	25.541	Measured
Motius Joint-Position Evaluator	Default	4,034	0.432	0.580	0.662	188.125	38.453	56.885	Measured

Motion-to-Text

Protocol	Samples	BLEU-4	ROUGE-L	CIDEr	BERT F1	R@1	R@2	R@3	Matching
HumanML3D M2T	4,400	-	-	-	-	-	-	-	-

Motion Representation

MotionGPT generates HumanML3D-263 features at 20 fps. Per frame:

Slice	Dim	Meaning
root_rot_vel	1	root angular velocity
root_lin_vel	2	root linear velocity in the horizontal plane
root_y	1	root height
ric_data	63	local joint positions
rot_data	126	local joint rotations in continuous 6D format
local_vel	66	local joint velocities
foot_contact	4	binary foot-contact labels

The VQ-VAE converts normalized HumanML3D features into discrete motion tokens. MotionGPT then treats those tokens as a language vocabulary item alongside text tokens.

Motius Components

Component	Path
Pipeline	motius.pipelines.motiongpt.MotionGPTPipeline
Bundle	motius.models.motiongpt.MotionGPTBundle
Runtime	motius.models.motiongpt.network.mGPT.archs

Only the inference-time MotionGPT modules required by the bundle are included in this public package.

Citation

@inproceedings{jiang2023motiongpt,
  title={MotionGPT: Human Motion as a Foreign Language},
  author={Jiang, Biao and Chen, Xin and Liu, Wen and Yu, Jingyi and Yu, Gang and Chen, Tao},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}