model card

README.md

@@ -3,4 +3,795 @@ license: other
 license_name: nvidia-internal-scientific-research-and-development-model-license
 license_link: >-
   https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-internal-scientific-research-and-development-model-license/
+library_name: dvlt
+pipeline_tag: image-to-3d
+tags:
+  - 3d-reconstruction
+  - depth-estimation
+  - camera-pose
+  - pointmap
+  - looped-transformer
+base_model: facebook/dinov2-base
 ---
+
+<div align="center">
+<h1>Déjà View: Looping Transformers for Multi-View 3D Reconstruction</h1>
+
+<a href="https://arxiv.org/abs/2605.30215"><img src="https://img.shields.io/badge/arXiv-2605.30215-b31b1b.svg" alt="arXiv"></a>
+<a href="https://research.nvidia.com/labs/dvl/projects/dvlt/"><img src="https://img.shields.io/badge/Project_Page-green" alt="Project Page"></a>
+<a href="https://research.nvidia.com/labs/dvl/projects/dvlt/"><img src="https://img.shields.io/badge/GitHub-Code-black?logo=github" alt="GitHub"></a>
+
+**[NVIDIA](https://www.nvidia.com/)** &nbsp;&nbsp;&nbsp; **[University of Modena and Reggio Emilia](https://www.unimore.it/)** &nbsp;&nbsp;&nbsp; **[ETH Zurich](https://ethz.ch/)**
+
+Alessandro Burzio, Tobias Fischer, Sven Elflein, Qunjie Zhou, Riccardo de Lutio, Jiawei Ren, Jiahui Huang, Shengyu Huang, Marc Pollefeys, Laura Leal-Taixé, Zan Gojcic, Haithem Turki
+</div>
+
+# Model Overview
+
+### Description
+
+Déjà View Looping Transformer (DVLT) is a feed-forward three-dimensional (3D) reconstruction model that takes unposed Red, Green, Blue (RGB) images or video and predicts per-pixel depth, ray maps (and thus 3D points), and per-view camera intrinsics/extrinsics in a single pass.
+
+**Novelty:** A weight-tied looped transformer — instead of stacking many distinct layers, a single shared block is applied for K refinement steps over a DINOv2-initialized per-view state, with each step conditioned on a continuous time interval (t_k, t_k+1) ⊂ [0, 1]. A single checkpoint exposes the iteration count K as an inference-time compute/quality knob without retraining separate models (released checkpoint valid for K ∈ [8, 16]).
+
+This model is for research and development only.
+
+### License/Terms of Use
+
+**Model.** The model (checkpoints, learned weights, and configuration files) is released under the **NVIDIA Internal Scientific Research and Development Model License**: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-internal-scientific-research-and-development-model-license/. The license grants a limited, non-exclusive, royalty-free right to use the model for internal scientific research and development in a non-production environment; redistribution, deployment in production, sublicensing, and use for generating works for sale or distribution are not permitted.
+
+**Source code.** The accompanying source code is licensed separately. The repository is primarily licensed under the **Apache License, Version 2.0** — see the `LICENSE` file at the repository root. Portions of the codebase derived from VGGT (Meta) are distributed under the VGGT License v1; the full text is provided in `LICENSES/VGGT-LICENSE.txt`. Full third-party attribution, per-file notices, and upstream license texts are collected in `THIRD_PARTY_LICENSES.md`.
+
+
+### Deployment Geography
+Global
+
+### Use Case
+
+Primary users:
+- Computer Vision Researchers: For benchmarking multi-view 3D reconstruction, studying weight-tied / recurrent transformer architectures, and developing neural rendering pipelines.
+- Augmented Reality/Virtual Reality & Robotics Engineers: For real-time simultaneous localization and mapping (SLAM), scene understanding, and navigation research prototypes.
+- 3D Content Creators: For rapid conversion of unposed video/image collections into 3D assets.
+
+Primary Use Cases:
+- 3D Reconstruction: Fast, feed-forward estimation of dense per-pixel depth, ray maps, and per-view camera poses from unposed images or video, without iterative optimization.
+- Structure-from-motion (SfM) Replacement: Accelerating initialization for 3D Gaussian Splatting and Neural Radiance Field (NeRF) training by replacing slow SfM pipelines (e.g., COLMAP).
+- Compute-Adaptive Inference: A single checkpoint supports a range of recurrence step counts (8–16) at inference, letting downstream applications trade reconstruction quality for latency without retraining.
+
+### Release Date
+GitHub 06/02/2026 via [URL TBD] <br>
+Hugging Face (HF) 06/02/2026 via https://huggingface.co/nvidia/dvlt <br>
+
+## References
+
+- [Pre-print](https://arxiv.org/abs/2605.30215)
+- Backbone: [DINOv2: Learning Robust Visual Features without Supervision](https://github.com/facebookresearch/dinov2)
+
+## Quick Start
+
+Install the [code](https://research.nvidia.com/labs/dvl/projects/dvlt/) (preferably in a conda environment):
+
+```bash
+conda create -n dvlt python=3.12 && conda activate dvlt
+conda install pytorch=2.5.1 torchvision pytorch-cuda=12.4 -c pytorch -c nvidia -c conda-forge
+pip install -e .[all]
+```
+
+Run feed-forward 3D reconstruction on a directory of images, a video, or an explicit list of frames:
+
+```python
+import torch
+from accelerate import Accelerator
+
+from dvlt.model.dvlt.model import DVLT
+from dvlt.util.preprocess import load_sequence, preprocess_images
+
+accelerator = Accelerator(mixed_precision="bf16")
+
+model = DVLT(img_size=504)
+model.load_pretrained("nvidia/dvlt")  # local dir, HTTPS URL, or HF Hub repo id
+model.setup_test(accelerator)
+
+# load_sequence accepts a directory, a single video, or an explicit list of files.
+_, frames = load_sequence("path/to/scene_dir")
+batch = preprocess_images(frames, img_size=504, patch_size=14, device=accelerator.device)
+
+with torch.no_grad(), accelerator.autocast():
+    predictions = model.predict(batch, accelerator)
+
+cameras = predictions["cameras"][0]             # Cameras object with shape [S]
+extrinsics_c2w = cameras.camera_to_worlds       # (S, 3, 4) — OpenCV convention [R | t]
+intrinsics = cameras.get_intrinsics_matrices()  # (S, 3, 3)
+depths = predictions["depths"][0]               # (S, H, W)
+world_points = predictions["world_points"][0]   # (S, H, W, 3)
+```
+
+## Model Architecture
+**Architecture Type:** Transformer <br>
+
+**Network Architecture:** Vision Transformer (ViT) with a single weight-tied looped block applied recurrently for K refinement steps. <br>
+
+**Backbone:** DINOv2 ViT-B (patch size 14).
+
+**Number of model parameters:** 1.17 * 10^8 <br>
+
+## Input(s)
+**Input Type(s):** Image, video <br>
+
+**Input Format(s):**
+- Image: Red, Green, Blue (RGB) <br>
+- Video: .mov, .mp4 (decoded to frames) <br>
+
+**Input Parameters:**
+- Image collection: Three-Dimensional (3D) — a sequence/stack of two-dimensional (2D) RGB images per scene. <br>
+- Video: Three-Dimensional (3D) — a sequence of two-dimensional (2D) frames per clip. <br>
+
+**Other Properties Related to Input:**
+- Video is decoded to frames at the video's native frame rate.
+- Training resolution: 504-pixel longest edge; sides padded to multiples of patch size 14.
+- Number of views per scene at training time: V ∈ [2, 18]; inference supports a wider range (memory-bound).
+
+## Output(s)
+
+**Output Type(s):**
+- Per-pixel depth map
+- Per-pixel ray map (origin + unnormalized direction)
+- Per-view camera parameters (extrinsics, intrinsics)
+
+**Output Format(s):**
+- Depth map: scalar metric distance per pixel
+- Ray map: 3D origin + 3D direction per pixel (6 channels)
+- Point cloud (derived analytically as X = R^o + D · R^d, i.e. ray origin plus depth times ray direction): X, Y, Z per pixel
+- Camera intrinsics: focal length, principal point
+- Camera extrinsics: rotation matrix (camera-to-world), translation vector
+
+**Output Parameters:**
+- Depth / confidence: Two-Dimensional (2D) per view (H × W) <br>
+- Ray map / point cloud: Three-Dimensional (3D) per view (H × W × 3) <br>
+- Camera intrinsics: Two-Dimensional (2D) (3 × 3) <br>
+- Camera extrinsics: Two-Dimensional (2D) (4 × 4) <br>
+
+**Other Properties Related to Output:**
+- One 3D point, one depth value, one ray, and one depth-confidence value are predicted per input pixel in each image.
+- One set of camera parameters (intrinsics + extrinsics) is predicted per input image, expressed in the coordinate frame of the first view.
+- Cameras are recovered from the predicted ray maps following [Lin et al. 2025](https://arxiv.org/abs/2510.21010); a camera multi-layer perceptron (MLP) head provides a faster auxiliary alternative.
+
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., Compute Unified Device Architecture (CUDA) libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
+
+## Software Integration
+**Runtime Engine(s):** PyTorch 2.5 (Hugging Face Accelerate)
+
+**Supported Hardware Microarchitecture Compatibility:** <br>
+* NVIDIA Ampere <br>
+* NVIDIA Blackwell <br>
+* NVIDIA Hopper <br>
+* NVIDIA Lovelace <br>
+
+
+**Preferred/Supported Operating System(s):** Linux
+
+The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment. <br>
+
+## Model Version(s)
+- v1.0: Initial release. DINOv2 ViT-B backbone (patch size 14), two-stage training (200,000 iterations stage 1 + 40,000 iterations stage-2 depth-decoder finetune) on 128 H100 GPUs, K ~ Beta(2, 1) on [8, 16].
+
+
+## Training, Testing, and Evaluation Datasets
+
+DVLT is trained on a mixture of 28 public datasets. Per-dataset realized mix shares (per-epoch frame-view sampling under temperature α = 0.5 on per-dataset frame counts) are listed inline below.
+
+### Training Datasets
+
+#### **Aria Synthetic Environments**
+
+**Link:** https://www.projectaria.com/datasets/aria-synthetic-environments/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 13.77% <br>
+
+**Properties:** Large-scale synthetic dataset of egocentric video sequences in realistic indoor environments, rendered with associated ground-truth camera trajectories, depth, semantic / instance segmentation, and 3D scene layout. <br>
+
+-----
+
+#### **Wild RGB-D**
+
+**Link:** https://github.com/wildrgbd/wildrgbd  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 9.32% <br>
+
+**Properties:** Large-scale dataset of ~20,000 RGB + Depth (RGB-D) videos of objects captured in the wild using iPhones. Includes camera poses and object masks. <br>
+
+-----
+
+#### **TRELLIS**
+
+**Link:** https://github.com/microsoft/TRELLIS  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 9.04% <br>
+
+**Properties:** Multi-view renderings of 3D objects produced via the TRELLIS rendering pipeline (Microsoft). We use 5 of the 6 TRELLIS subsets — **3D-FUTURE**, **ABO** (Amazon Berkeley Objects), **HSSD** (Habitat Synthetic Scenes Dataset), **Toys4K**, and the **Objaverse-XL Sketchfab** subset — and explicitly exclude the Objaverse-XL GitHub subset. Each upstream asset collection retains its own license. <br>
+
+-----
+
+#### **Common Objects in 3D (CO3D)**
+
+**Link:** https://ai.meta.com/datasets/co3d-dataset/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 8.84% <br>
+
+**Properties:** ~6 million images. Large-scale dataset of object-centric videos captured with smartphones, annotated with camera poses and point clouds via structure-from-motion. <br>
+
+-----
+
+#### **Taskonomy**
+
+**Link:** http://taskonomy.stanford.edu/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 7.26% <br>
+
+**Properties:** ~4 million indoor images across 600 buildings, captured with structured-light depth sensors. Provides per-pixel depth, surface normals, semantic / instance segmentation, and camera poses. Loaded via the [OmniData](https://github.com/EPFL-VILAB/omnidata) multi-view preprocessing pipeline. <br>
+
+-----
+
+#### **ScanNet**
+
+**Link:** http://www.scan-net.org/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Hybrid: Automatic/Sensors, Human] <br>
+
+**Mix share:** 6.23% <br>
+
+**Properties:** RGB-D video dataset containing 2.5 million views in more than 1,500 scans, annotated with 3D camera poses, surface reconstructions, and instance-level semantic segmentations. Captured with an iPad-mounted depth sensor and crowdsourced semantic annotation. <br>
+
+-----
+
+#### **DL3DV-10K**
+
+**Link:** https://dl3dv-10k.github.io/DL3DV-10K/  <br>
+
+**Data Modality**: Image, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 5.84% <br>
+
+**Properties:** Large-scale scene dataset containing over 10,000 videos of diverse real-world scenes captured for deep-learning 3D vision. Camera poses are obtained using structure-from-motion. <br>
+
+-----
+
+#### **Cubify Anything**
+
+**Link:** https://github.com/apple/ml-cubifyanything  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Mix share:** 5.73% <br>
+
+**Properties:** Aligns handheld RGB-D captures from ARKitScenes with laser scans, creating a large-scale labeled dataset for 3D object detection. <br>
+
+-----
+
+#### **TartanAir V2**
+
+**Link:** https://tartanair.org/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 4.78% <br>
+
+**Properties:** Updated, photorealistic synthetic SLAM dataset with diverse environments, motion patterns, and lighting/weather conditions. Provides RGB, depth, and camera trajectories. <br>
+
+-----
+
+#### **Parallel Domain 4D**
+
+**Link:** https://github.com/basilevh/gcd?tab=readme-ov-file#paralleldomain-4d  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 4.75% <br>
+
+**Properties:** Synthetic video dataset rendered using the [Parallel Domain](https://paralleldomain.com/) synthetic data engine, designed for dynamic novel-view synthesis and 4D reconstruction. Dataset assembled and released by GCD (Van Hoorick et al., ECCV 2024). <br>
+
+-----
+
+#### **ScanNet++**
+
+**Link:** https://scannetpp.mlsg.cit.tum.de/scannetpp/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Mix share:** 3.96% <br>
+
+**Properties:** Millions of frames across ~460 scenes. High-resolution real-world 3D indoor semantic scene understanding dataset using laser scanning and DSLR/iPhone capture. The evaluation set uses a clean scene-level split disjoint from the training scenes. <br>
+
+-----
+
+#### **Mid-Air**
+
+**Link:** https://midair.ulg.ac.be/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 2.61% <br>
+
+**Properties:** ~420,000 video frames. Synthetic dataset for low-altitude drone flight, featuring multi-modal data (RGB, depth, normals, stereo) across different climate conditions. <br>
+
+-----
+
+#### **Dynamic Replica**
+
+**Link:** https://dynamic-stereo.github.io/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 2.16% <br>
+
+**Properties:** 145,200 stereo frames (from 524 videos). Synthetic rendered videos of scanned environments with dynamic objects (people / animals); includes stereo images, disparity maps, and camera parameters. <br>
+
+-----
+
+#### **TartanAir**
+
+**Link:** https://theairlab.org/tartanair-dataset/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 2.13% <br>
+
+**Properties:** Challenging visual SLAM dataset collected in photorealistic simulation environments with various lighting / weather conditions. <br>
+
+-----
+
+#### **BEDLAM**
+
+**Link:** https://bedlam.is.tue.mpg.de/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 1.99% <br>
+
+**Properties:** Synthetic dataset of moving human bodies in 3D environments (~10K sequences), with photorealistic rendering and ground-truth depth / camera parameters. <br>
+
+-----
+
+#### **BlendedMVG**
+
+**Link:** https://github.com/YoYo000/BlendedMVS  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 1.35% <br>
+
+**Properties:** Higher-resolution / extended-scenes variant from the BlendedMVS family. 3D models are reconstructed from real images, then rendered to create synthetic training pairs blended with the original images. <br>
+
+-----
+
+#### **MegaDepth**
+
+**Link:** https://www.cs.cornell.edu/projects/megadepth/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Automated] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 1.16% <br>
+
+**Properties:** ~150,000 reconstructed images derived from internet photo collections (Flickr) using structure-from-motion (SfM) and multi-view stereo (MVS) to generate depth maps. <br>
+
+-----
+
+#### **Replica**
+
+**Link:** https://github.com/facebookresearch/Replica-Dataset  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 0.95% <br>
+
+**Properties:** 18 high-fidelity reconstructions of real-world indoor spaces (apartments, hotel rooms, offices) captured with a custom RGB-D capture rig and dense surface reconstructions. Loaded via the [OmniData](https://github.com/EPFL-VILAB/omnidata) multi-view preprocessing pipeline. <br>
+
+-----
+
+#### **Virtual KITTI 2**
+
+**Link:** https://europe.naverlabs.com/proxy-virtual-worlds-vkitti-2/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.83% <br>
+
+**Properties:** Photorealistic synthetic video dataset mimicking the real KITTI dataset, used for object detection, tracking, and depth estimation. <br>
+
+-----
+
+#### **Hypersim**
+
+**Link:** https://github.com/apple/ml-hypersim  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.69% <br>
+
+**Properties:** ~77,400 images. Photorealistic synthetic dataset for indoor scene understanding. Contains RGB, depth, semantic segmentation, and camera parameters. <br>
+
+-----
+
+#### **Kubric**
+
+**Link:** https://github.com/google-research/kubric  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.60% <br>
+
+**Properties:** A data-generation pipeline using PyBullet and Blender to create photo-realistic synthetic videos with rich annotations. We use the dataset built from assets from [Google Scanned Objects](https://research.google/blog/scanned-objects-by-google-research-a-dataset-of-3d-scanned-common-household-items/) and environments from [HDRI Haven](https://polyhaven.com/hdris). <br>
+
+-----
+
+#### **GTA-SfM**
+
+**Link:** https://github.com/lwwang/GTA-SfM-toolkit  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.52% <br>
+
+**Properties:** Photorealistic synthetic dataset derived from GTA-V environments, providing ground-truth depth, camera poses, and dense SfM-style supervision in urban scenes. <br>
+
+-----
+
+#### **MatrixCity**
+
+**Link:** https://city-super.github.io/matrixcity/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.49% <br>
+
+**Properties:** Large-scale synthetic city dataset for neural rendering, generated using Unreal Engine 5. Both the aerial small-city-block subset and the street-level subset are used. <br>
+
+-----
+
+#### **Mapillary Planet-Scale Depth dataset**
+
+**Link:** https://www.mapillary.com/dataset/depth  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Mix share:** 0.49% <br>
+
+**Version:** v1.1 <br>
+
+**Properties:** ~750,000 images. Dense depth dataset derived from street-level imagery using SfM, covering a wide variety of locations and camera models. <br>
+
+-----
+
+#### **Mapillary Metropolis**
+
+**Link:** https://www.mapillary.com/dataset/metropolis  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Mix share:** 0.47% <br>
+
+**Version:** v1 <br>
+
+**Properties:** ~28,000 images. City-scale dataset containing street-level panoramic images, LiDAR point clouds, and aerial imagery. <br>
+
+-----
+
+#### **UnrealStereo4K**
+
+**Link:** https://github.com/fabiotosi92/SMD-Nets#unrealstereo4k  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.44% <br>
+
+**Properties:** High-resolution synthetic stereo dataset generated using Unreal Engine 4. <br>
+
+-----
+
+#### **MVS-Synth**
+
+**Link:** https://phuang17.github.io/DeepMVS/mvs-synth.html  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.44% <br>
+
+**Properties:** Photorealistic synthetic urban dataset, rendered from GTA-V, with ground-truth depth and camera poses for multi-view stereo. <br>
+
+-----
+
+#### **Spring Benchmark**
+
+**Link:** https://spring-benchmark.org/  <br>
+
+**Data Modality**: Image, Depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Synthetic] <br>
+
+**Labeling Method by dataset** [Synthetic] <br>
+
+**Mix share:** 0.40% <br>
+
+**Properties:** High-resolution photo-realistic computer-generated imagery (Blender) for optical flow and stereo benchmarking. <br>
+
+
+### Testing / Evaluation Datasets
+
+DVLT is evaluated (and tested) on five public reconstruction benchmarks spanning indoor, outdoor, object-centric, and driving scenes. We use each benchmark's standard held-out test split as both the validation/evaluation set during development and the final testing set for the released checkpoint; no separate internal testing dataset is held out.
+
+#### **DTU**
+
+**Link:** https://roboimagedata.compute.dtu.dk/?page_id=36  <br>
+
+**Data Modality**: Image, depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Properties:** An industrial robot arm was mounted with a structured-light scanner. This allowed for structured-light scans corresponding to each image in the dataset. Images were taken by one of the cameras in the structured-light scanner. <br>
+
+---
+
+#### **ETH3D Stereo Benchmark**
+
+**Link:** https://www.eth3d.net/overview  <br>
+
+**Data Modality**: Image, depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Properties:** The multi-view stereo / 3D reconstruction benchmark covers a variety of indoor and outdoor scenes. Ground-truth geometry has been obtained using a high-precision laser scanner. A DSLR camera and a synchronized multi-camera rig with varying field of view were used to capture images. <br>
+
+---
+
+#### **7-Scenes**
+
+**Link:** https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/  <br>
+
+**Data Modality**: Image, depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Automatic/Sensors] <br>
+
+**Properties:** 7-Scenes is an RGB-D dataset featuring seven indoor scenes captured with a Kinect camera. The dataset includes RGB images (640 × 480), depth maps, and camera-to-world poses. <br>
+
+---
+
+#### **ScanNet++**
+
+**Link:** https://scannetpp.mlsg.cit.tum.de/scannetpp/  <br>
+
+**Data Modality**: Image, depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [1 Million to 1 Billion Images] <br>
+
+**Data Collection Method by dataset** [Human] <br>
+
+**Labeling Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Properties:** High-resolution real-world 3D indoor semantic scene understanding dataset using laser scanning and DSLR / iPhone capture. ScanNet++ also serves as a training dataset for multiple baselines and for DVLT; we enforce a clean scene-level split between the training and evaluation data. <br>
+
+---
+
+#### **nuScenes**
+
+**Link:** https://www.nuscenes.org/  <br>
+
+**Data Modality**: Image, depth, Camera extrinsics and intrinsics.
+
+**Image Training Data Size** [Less than a Million Images] <br>
+
+**Data Collection Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Labeling Method by dataset** [Hybrid: Human, Automatic/Sensors] <br>
+
+**Properties:** Autonomous-driving dataset of 1,000 scenes collected in Boston and Singapore with a full sensor suite (6 cameras, 1 LiDAR, 5 radars, GPS, IMU). Provides synchronized multi-view imagery, LiDAR-derived depth, and ego-poses, used here as the driving-domain evaluation benchmark. <br>
+
+
+# Inference
+**Acceleration Engine:** PyTorch native (bf16 via Hugging Face Accelerate) <br>
+**Test Hardware:** NVIDIA H100, NVIDIA A100 <br>
+
+## Ethical Considerations
+NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>
+
+Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included. <br>
+
+Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).  <br>
+
+## Citation
+
+If you find this work useful, please cite:
+
+```bibtex
+@article{burzio2026dejaview,
+  title   = {D\'ej\`a View: Looping Transformers for Multi-View 3D Reconstruction},
+  author  = {Burzio, Alessandro and Fischer, Tobias and Elflein, Sven and Zhou, Qunjie and de Lutio, Riccardo and Ren, Jiawei and Huang, Jiahui and Huang, Shengyu and Pollefeys, Marc and Leal-Taix{\'e}, Laura and Gojcic, Zan and Turki, Haithem},
+  journal = {arXiv preprint arXiv:2605.30215},
+  year    = {2026}
+}
+```