README.md

---
license: other
license_name: nvidia-oneway-noncommercial
license_link: https://github.com/nv-tlabs/cmd/blob/main/LICENSE
pipeline_tag: image-to-video
tags:
  - video-generation
  - image-to-video
  - world-model
  - diffusion
  - autoregressive
  - camera-control
  - safetensors
---

# Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation

**Hmrishav Bandyopadhyay<sup>1,2</sup>, Xuanchi Ren<sup>1</sup>, Zijian Huang<sup>1</sup>, Jay Zhangjie Wu<sup>1</sup>, Tianshi Cao<sup>1</sup>, Ruilong Li<sup>1</sup>, Bryan Chu<sup>1</sup>, Sanja Fidler<sup>1</sup>, Yi-Zhe Song<sup>2</sup>, Zian Wang<sup>1</sup>**

<sup>1</sup>NVIDIA  
<sup>2</sup>SketchX, CVSSP, University of Surrey

[Project Page](https://hmrishavbandy.github.io/cmd-site/) | [Code](https://github.com/nv-tlabs/cmd)

## Model Overview

### Description

Context-Matched Distillation (CMD) is a method for autoregressive video distillation using causal teachers. This repository provides CMD checkpoints based on Cosmos-Predict2.5 2B for causal image-to-video generation.

The release includes chunk-1 and chunk-4 models for short- and long-duration generation, together with camera-control variants. The models use local temporal attention and generate video autoregressively in causal chunks.

This model is for research and development only.

### License and Terms of Use

This model is released under the [NVIDIA OneWay Noncommercial License](https://github.com/NVIDIA/generative-model-design-license/blob/main/NVIDIA%20OneWay%20Noncommercial%20License.md). It may only be used for non-commercial research or educational purposes.

### Deployment Geography

Global

### Intended Use

Primary users include researchers and developers studying:

- Autoregressive video generation.
- Causal video distillation.
- Long-duration video generation.
- Interactive world models.
- Camera-controlled video generation.

The checkpoints are not intended for safety-critical applications or uses prohibited by the model license.

## Available Checkpoints

| Checkpoint | Configuration | Intended use |
|---|---|---|
| `chunk1_short_t24_l21.safetensors` | `t24_l21` | Chunk-1 short-duration generation |
| `chunk4_short_t21_l16.safetensors` | `t21_l16` | Chunk-4 short-duration generation |
| `chunk1_long_t126_l21.safetensors` | `t126_l21` | Chunk-1 long-duration generation |
| `chunk4_long_t121_l16.safetensors` | `t121_l16` | Chunk-4 long-duration generation |
| `chunk1_camera_control_t32_l21.safetensors` | `t32_l21` | Chunk-1 generation with camera control |
| `chunk4_camera_control_t29_l24.safetensors` | `t29_l24` | Chunk-4 generation with camera control |

In the configuration names, `t` denotes frame duration and `l` denotes local attention. Checkpoint names do not include dataset names or training iteration information.

## Quick Start

Install the reference implementation:

```bash
git clone https://github.com/nv-tlabs/cmd.git
cd cmd

conda create -n causal-cosmos python=3.10 -y
conda activate causal-cosmos
python -m pip install -r requirements.txt
python -m pip install flash-attn --no-build-isolation
python setup.py develop
```

Download the checkpoints:

```bash
hf download nvidia/cmd --local-dir checkpoints
```

To download one model only:

```bash
hf download nvidia/cmd \
  chunk1_short_t24_l21.safetensors \
  --local-dir checkpoints
```

Follow the inference instructions in the [CMD repository](https://github.com/nv-tlabs/cmd) to select the matching model configuration and run generation.

## Model Architecture

Architecture type: Causal video diffusion transformer

Base model: [Cosmos-Predict2.5 2B](https://huggingface.co/nvidia/Cosmos-Predict2.5-2B)

Generation strategy: Block-autoregressive video generation

CMD uses causal temporal attention to generate video sequentially. Chunk-1 and chunk-4 variants provide different autoregressive generation granularities, while the camera-control variants add camera-motion conditioning.

## Inputs

Standard checkpoints accept:

- An initial image.
- A text prompt.

Camera-control checkpoints additionally accept a camera trajectory in the format expected by the CMD inference pipeline.

## Outputs

The model produces an RGB video conditioned on the initial image, text prompt, and, when applicable, camera trajectory. Output duration is determined by the selected `t` configuration.

## Software Integration

- Framework: PyTorch
- Checkpoint format: Safetensors
- Reference implementation: [nv-tlabs/cmd](https://github.com/nv-tlabs/cmd)
- Preferred operating system: Linux
- Recommended hardware: NVIDIA CUDA-capable GPU

The checkpoints must be used with the corresponding model architecture and configuration supplied by the reference implementation.

## Training Method

CMD uses a two-stage procedure:

1. Causal teacher pretraining.
2. Context-matched distillation.

See the [CMD repository](https://github.com/nv-tlabs/cmd) and [project page](https://hmrishavbandy.github.io/cmd-site/) for the method and evaluation details.

## Limitations

- Autoregressive errors may accumulate during long rollouts.
- Visual quality and temporal consistency may degrade over time.
- Generated motion may be physically implausible.
- Fine details, text, faces, hands, and rapid motion may contain artifacts.
- Prompt adherence may vary with scene complexity.
- Camera-control outputs may deviate from the requested trajectory.
- Generated content may reflect biases present in the training data.

Users should evaluate the model on data representative of their intended application before deployment.

## Ethical Considerations

Generated video may be mistaken for authentic footage. Users should clearly identify synthetic content where appropriate and must not use the model for deception, impersonation, harassment, misinformation, or violations of privacy, intellectual-property, or other applicable rights.

NVIDIA believes Trustworthy AI is a shared responsibility. Developers should ensure that their use of this model complies with applicable laws, policies, licenses, and safety requirements.

Please report model quality, risk, security vulnerabilities, or NVIDIA AI concerns through the [NVIDIA support portal](https://www.nvidia.com/en-us/support/submit-a-case/).

## Citation

If you find this work useful, please cite:

```bibtex
@article{bandyopadhyay2026context,
  title   = {Context-Matched Distillation: Teacher Causality for Autoregressive Video Distillation},
  author  = {Bandyopadhyay, Hmrishav and Ren, Xuanchi and Huang, Zijian
             and Wu, Jay Zhangjie and Cao, Tianshi and Li, Ruilong
             and Chu, Bryan and Fidler, Sanja and Song, Yi-Zhe
             and Wang, Zian},
  journal = {arXiv preprint arXiv:2608.13391},
  year    = {2026},
  eprint  = {2608.13391},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url     = {https://arxiv.org/abs/2608.13391}
}
```

## Acknowledgements

This project builds on [Self-Forcing](https://github.com/guandeh17/Self-Forcing) and [NVIDIA Cosmos-Predict2.5](https://github.com/nvidia-cosmos/cosmos-predict2.5).