File size: 3,832 Bytes
0c82bc2 c3cf3d8 0c82bc2 c3cf3d8 a344971 c3cf3d8 a344971 c3cf3d8 a344971 c3cf3d8 affa705 a344971 c3cf3d8 affa705 c3cf3d8 affa705 3b79e46 c3cf3d8 affa705 c3cf3d8 affa705 c3cf3d8 affa705 c3cf3d8 affa705 c3cf3d8 6d398f0 c3cf3d8 affa705 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
license: other
license_name: ltx-2-community
tags:
- video-generation
- audio
- lora
- temporal-control
- text-to-video
---
<div align="center">
<h1>The Missing Temporal Link: Temporal Context Routing<br>for Script-Driven Audio-Video Generation</h1>
<p>
Yichen Liu<sup>1</sup>, Quanwei Zhang<sup>2</sup>, Haozhe Wang<sup>3</sup>, Donghao Zhou<sup>4</sup>, Xiaojie Li<br>
Yang Shi<sup>2</sup>, Jiaming Liu<sup>2</sup>, Ruihua Huang<sup>2</sup>, Yingtian Zou<sup>5</sup>, Daquan Zhou<sup>1</sup>
</p>
<p>
<sup>1</sup> Peking University · <sup>2</sup> Qwen Applications<br>
<sup>3</sup> HKUST · <sup>4</sup> CUHK · <sup>5</sup> Shanghai Jiao Tong University
</p>
[](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/)
[](https://github.com/DAGroup-PKU/Temporal-Context-Routing)
[](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/temporal-context-routing.safetensors)
[](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/test_prompts_200.json)
</div>
**Temporal Context Routing (TCR)** aligns script-specified shot and dialogue timing with the shared video-audio timeline. Timing bypasses the text encoder and enters cross-attention logits as a routing score, separating *what* to generate from *when* it should appear. Built on LTX-2.3 22B, TCR reduces shot-boundary error from **1.11 s to 0.042 s**—about one frame at 24 fps—and raises dialogue timing accuracy from **28.3% to 84.1%**.
This repository hosts the released **LoRA** (`temporal-context-routing.safetensors`, ~2.3 GB) and the **200 held-out test prompts** (`test_prompts_200.json`) used in the paper. Training and inference code is at [DAGroup-PKU/Temporal-Context-Routing](https://github.com/DAGroup-PKU/Temporal-Context-Routing). Demos are available on the [project page](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/).
## Files
| File | What it is |
| --- | --- |
| `temporal-context-routing.safetensors` | TCR LoRA, rank 128, trained on LTX-2.3 22B (paper 7k-step recipe) |
| `test_prompts_200.json` | 200 held-out script JSON files with per-shot / per-line `time_range` |
You still need the **LTX-2.3 22B** checkpoint and the **Gemma 3 12B** text encoder locally. One 80 GB GPU is the working setup.
```bash
huggingface-cli download starry0929/Temporal-Context-Routing \
temporal-context-routing.safetensors test_prompts_200.json \
--local-dir ./weights
```
Then follow the [code README](https://github.com/DAGroup-PKU/Temporal-Context-Routing#inference):
```bash
bash scripts/infer.sh \
--checkpoint /path/to/ltx-2.3-22b-dev.safetensors \
--text-encoder-path /path/to/gemma-3-12b-it \
--lora-path ./weights/temporal-context-routing.safetensors \
--output outputs/tcr_infer.mp4
```
## Citation
```bibtex
@inproceedings{liu2027missingtemporallink,
title = {The Missing Temporal Link: Temporal Context Routing
for Script-Driven Audio-Video Generation},
author = {Liu, Yichen and Zhang, Quanwei and Wang, Haozhe and
Zhou, Donghao and Li, Xiaojie and Shi, Yang and Liu, Jiaming and
Huang, Ruihua and Zou, Yingtian and Zhou, Daquan},
booktitle = {Under review},
year = {2027}
}
```
## License
The LoRA is trained on LTX-2.3 and is intended to be used with that backbone under the [LTX-2 Community License](https://github.com/DAGroup-PKU/Temporal-Context-Routing/blob/main/LICENSE).
|