IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer

Project Page | Paper | Code

IGGT4D is a streaming instance-grounded geometry Transformer for online 4D scene understanding. It processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally maintains a unified representation of camera motion, scene geometry, and persistent object identity. This enables scalable feed-forward reconstruction of long, dynamic sequences with consistent geometry and instance-level understanding.

Highlights

• Unified geometry-instance reconstruction: jointly models camera motion, scene geometry, and temporally consistent instance features.
• Streaming inference: processes frames causally and updates the scene representation online for long video sequences.
• Persistent object identities: uses lightweight streaming instance clustering to associate local masks with global instance IDs.
• InsScene4D-147K: introduces a large-scale collection of real/synthetic and static/dynamic sequences with RGB, depth, poses, and temporally consistent instance masks.
• Downstream applications: supports 3D/4D reconstruction, pose estimation, instance spatial tracking, open-vocabulary segmentation, and LMM-based 4D scene grounding.

Release Status

The model checkpoint is hosted in this repository. Please follow the GitHub repository and project page for code, dataset, benchmark, and usage updates.

Citation

If you find IGGT4D useful in your research, please cite:

@article{zou2026iggt4d,
  title   = {IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer},
  author  = {Zou, Zhengyu and Li, Hao and Jiao, Kuixuan and Liu, Liu and Xiao, Tingyang and Zhou, Xiaolin and Hong, Fangzhou and Su, Zhizhong and Zhang, Dingwen and Liu, Ziwei},
  journal = {arXiv preprint arXiv:2607.19228},
  year    = {2026}
}

License

The checkpoint is released under the Apache License 2.0.