| # PyraTok: Language-Aligned Pyramidal Tokenizer for Video Understanding and Generation | |
| <div align="center"> | |
| <a href="https://plan-lab.github.io/pyratok"><img src="https://img.shields.io/badge/Project-Website-blue?style=for-the-badge&logo=googlechrome"></a> | |
| <a href="https://arxiv.org/abs/2601.16210"><img src="https://img.shields.io/badge/arXiv-2601.16210-b31b1b.svg?style=for-the-badge"></a> | |
| <a href="https://github.com/PLAN-Lab/PyraTok"><img src="https://img.shields.io/badge/Code-GitHub-black?style=for-the-badge&logo=github"></a> | |
| </div> | |
| --- | |
| ### 📢 Official Announcement | |
| **PyraTok** has been officially accepted to **CVPR 2026**! 🎉 | |
| This repository contains the pretrained weights and model implementation for the Language-aligned Pyramidal Tokenizer. | |
| --- | |
| ## 🚀 Overview | |
| **PyraTok** is a state-of-the-art video tokenizer that bridges the gap between video understanding and generation. Unlike traditional VAEs that operate at a single visual scale, PyraTok introduces a **Language-aligned Pyramidal Quantization (LaPQ)** module. | |
| ### Key Innovations: | |
| * **Pyramidal Structure:** Learns semantically structured discrete latents across multiple spatiotemporal resolutions. | |
| * **Language Alignment:** Tightly couples visual tokens with language using a shared, large binary codebook (up to 48K tokens). | |
| * **Scalability:** Robustly scales from standard resolutions to **4K/8K video** processing. | |
| * **Unified Backbone:** A single model that excels in Video QA, Zero-Shot Segmentation, and high-fidelity Text-to-Video generation. | |
| ``` | |
| @inproceedings{susladkar2026pyratok, | |
| title={PyraTok: Language-Aligned Pyramidal Tokenizer for Video Understanding and Generation}, | |
| author={Susladkar, Onkar and Prakash, Tushar and Juvekar, Adheesh and Nguyen, Kiet A. and Jang, Dong-Hwan and Dhillon, Inderjit S. and Lourentzou, Ismini}, | |
| booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, | |
| year={2026} | |
| } | |
| ``` |