README.md

---
license: other
license_name: ltx-2-community
tags:
  - video-generation
  - audio
  - lora
  - temporal-control
  - text-to-video
---

<div align="center">

<h1>The Missing Temporal Link: Temporal Context Routing<br>for Script-Driven Audio-Video Generation</h1>

<p>
  Yichen Liu<sup>1</sup>, Quanwei Zhang<sup>2</sup>, Haozhe Wang<sup>3</sup>, Donghao Zhou<sup>4</sup>, Xiaojie Li<br>
  Yang Shi<sup>2</sup>, Jiaming Liu<sup>2</sup>, Ruihua Huang<sup>2</sup>, Yingtian Zou<sup>5</sup>, Daquan Zhou<sup>1</sup>
</p>

<p>
  <sup>1</sup>&nbsp;Peking University&nbsp;&nbsp;&middot;&nbsp;&nbsp;<sup>2</sup>&nbsp;Qwen Applications<br>
  <sup>3</sup>&nbsp;HKUST&nbsp;&nbsp;&middot;&nbsp;&nbsp;<sup>4</sup>&nbsp;CUHK&nbsp;&nbsp;&middot;&nbsp;&nbsp;<sup>5</sup>&nbsp;Shanghai Jiao Tong University
</p>

[![Project Page](https://img.shields.io/badge/Project-Page-4d86c4)](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/)
[![GitHub](https://img.shields.io/badge/Code-GitHub-24292f?logo=github)](https://github.com/DAGroup-PKU/Temporal-Context-Routing)
[![LoRA Weights](https://img.shields.io/badge/LoRA-Weights-ffbd2e?logo=huggingface)](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/temporal-context-routing.safetensors)
[![Test Prompts](https://img.shields.io/badge/Test-Prompts-2da44e)](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/test_prompts_200.json)

</div>

**Temporal Context Routing (TCR)** aligns script-specified shot and dialogue timing with the shared video-audio timeline. Timing bypasses the text encoder and enters cross-attention logits as a routing score, separating *what* to generate from *when* it should appear. Built on LTX-2.3 22B, TCR reduces shot-boundary error from **1.11 s to 0.042 s**—about one frame at 24 fps—and raises dialogue timing accuracy from **28.3% to 84.1%**.

This repository hosts the released **LoRA** (`temporal-context-routing.safetensors`, ~2.3 GB) and the **200 held-out test prompts** (`test_prompts_200.json`) used in the paper. Training and inference code is at [DAGroup-PKU/Temporal-Context-Routing](https://github.com/DAGroup-PKU/Temporal-Context-Routing). Demos are available on the [project page](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/).

## Files

| File | What it is |
| --- | --- |
| `temporal-context-routing.safetensors` | TCR LoRA, rank 128, trained on LTX-2.3 22B (paper 7k-step recipe) |
| `test_prompts_200.json` | 200 held-out script JSON files with per-shot / per-line `time_range` |

You still need the **LTX-2.3 22B** checkpoint and the **Gemma 3 12B** text encoder locally. One 80 GB GPU is the working setup.

```bash
huggingface-cli download starry0929/Temporal-Context-Routing \
  temporal-context-routing.safetensors test_prompts_200.json \
  --local-dir ./weights
```

Then follow the [code README](https://github.com/DAGroup-PKU/Temporal-Context-Routing#inference):

```bash
bash scripts/infer.sh \
  --checkpoint /path/to/ltx-2.3-22b-dev.safetensors \
  --text-encoder-path /path/to/gemma-3-12b-it \
  --lora-path ./weights/temporal-context-routing.safetensors \
  --output outputs/tcr_infer.mp4
```

## Citation

```bibtex
@inproceedings{liu2027missingtemporallink,
  title     = {The Missing Temporal Link: Temporal Context Routing
               for Script-Driven Audio-Video Generation},
  author    = {Liu, Yichen and Zhang, Quanwei and Wang, Haozhe and
               Zhou, Donghao and Li, Xiaojie and Shi, Yang and Liu, Jiaming and
               Huang, Ruihua and Zou, Yingtian and Zhou, Daquan},
  booktitle = {Under review},
  year      = {2027}
}
```

## License

The LoRA is trained on LTX-2.3 and is intended to be used with that backbone under the [LTX-2 Community License](https://github.com/DAGroup-PKU/Temporal-Context-Routing/blob/main/LICENSE).