Make the WorldDiT model card demo first (#1)

- Make the WorldDiT model card demo first (2884ee74ce9a6a114da7dd214a296986814be174)

README.md

@@ -1,19 +1,46 @@
 ---
 library_name: pytorch
 pipeline_tag: robotics
+language:
+  - en
 tags:
   - worlddit
+  - world-action-model
+  - world-models
   - libero
   - robot-learning
+  - robotic-manipulation
   - imitation-learning
+  - diffusion-transformer
   - diffusion-policy
+  - flow-matching
+inference: false
+widget:
+  - example_title: "LIBERO Spatial, task 5"
+    text: "Successful rollout, front view."
+    output:
+      url: "https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4"
+  - example_title: "LIBERO Object, task 8"
+    text: "Successful rollout, agent view."
+    output:
+      url: "https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4"
+  - example_title: "LIBERO Goal, task 10"
+    text: "Successful rollout, side view."
+    output:
+      url: "https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4"
+  - example_title: "LIBERO Long, task 6"
+    text: "Successful rollout, front view."
+    output:
+      url: "https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4"
 ---
 
 <p align="center">
   <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
 </p>
 
-<h1 align="center">WorldDiT</h1>
+# WorldDiT
+
+## One diffusion backbone learns what to do and what comes next.
 
 <p align="center">
   <a href="https://huggingface.co/bageldotcom/worlddit" target="_blank">
@@ -24,66 +51,82 @@ tags:
   </a>
 </p>
 
-WorldDiT is a diffusion-transformer policy for language-conditioned robotic
-manipulation. This public release provides suite-specific checkpoints for all
-four LIBERO benchmark suites together with a compact, self-contained inference
-and evaluation runtime.
+WorldDiT learns continuous robot action chunks and a future visual target
+through one shared diffusion transformer. Deployment keeps only the action
+path.
 
-The runtime is intentionally minimal: `inference.py` constructs the policy and
-loads a checkpoint, while `eval.py` performs headless single- or multi-GPU
-LIBERO evaluation.
+This release includes four LIBERO checkpoints, a self contained inference
+runtime, and an evaluator for reproducing the reported suite results.
 
-# Results
+## See WorldDiT act
 
-All results use 10 tasks × 50 episodes with the released evaluation protocol.
-The public runtime and checkpoints were revalidated from a clean installation
-across 2,000 episodes on 8xRTX Pro 6000 Blackwell GPUs.
+The four clips below show successful rollouts from the released checkpoints.
+Each clip covers a different LIBERO suite and camera view.
 
-| Suite | Successes | Success rate |
-|---|---:|---:|
-| LIBERO-Spatial | 490/500 | **98.0%** |
-| LIBERO-Object | 485/500 | **97.0%** |
-| LIBERO-Goal | 464/500 | **92.8%** |
-| LIBERO-10 | 459/500 | **91.8%** |
+<Gallery />
 
-# Key Characteristics
+| Suite | View | Video |
+|---|---|---|
+| LIBERO Spatial | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4) |
+| LIBERO Object | Agent view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4) |
+| LIBERO Goal | Side view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4) |
+| LIBERO Long | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4) |
 
-- Three-frame observation context
-- Seven-step action prediction horizon
-- Three actions executed between policy replans
-- Temporally ensembled action predictions
-- Suite-specific checkpoints for all four LIBERO benchmarks
-- Headless evaluation on one or more GPUs
-- Compact two-file inference and evaluation runtime
+## What is in this release
 
----
+| Release component | Included artifact |
+|---|---|
+| LIBERO Spatial policy | SafeTensors checkpoint |
+| LIBERO Object policy | SafeTensors checkpoint |
+| LIBERO Goal policy | SafeTensors checkpoint |
+| LIBERO Long policy | SafeTensors checkpoint |
+| Model runtime | `inference.py` |
+| Evaluation runtime | `eval.py` |
+| Frozen encoders | CLIP ViT B 32 and MAE ViT B |
+| Configuration | `config.json` |
+| Environment | Pinned Python requirements |
 
-# What This Repository Contains
+The repository is self contained for WorldDiT inference. LIBERO still provides
+the benchmark environments, assets, task definitions, and initial states.
 
-```text
-.
-├── checkpoints/
-│   ├── libero_10/model.safetensors
-│   ├── libero_goal/model.safetensors
-│   ├── libero_object/model.safetensors
-│   └── libero_spatial/model.safetensors
-├── dependencies/
-│   ├── ViT-B-32.pt
-│   └── mae_pretrain_vit_base.pth
-├── eval.py
-├── inference.py
-├── config.json
-└── requirements.txt
-```
+## Reported LIBERO results
 
-`dependencies/` contains the frozen visual and language encoder weights needed
-by the released policy. No additional model downloads are required.
+Across the four released suite checkpoints, WorldDiT records 1,898 successful
+episodes out of 2,000 under the selection aware evaluation protocol.
 
----
+| Suite | Successful episodes | Success rate |
+|---|---:|---:|
+| LIBERO Spatial | 490 of 500 | 98.0 percent |
+| LIBERO Object | 485 of 500 | 97.0 percent |
+| LIBERO Goal | 464 of 500 | 92.8 percent |
+| LIBERO Long | 459 of 500 | 91.8 percent |
+| Selection aware mean | 1,898 of 2,000 | 94.9 percent |
+
+The released runtime and checkpoints were revalidated from a clean installation
+on eight RTX Pro 6000 Blackwell GPUs.
+
+The result is selection aware because three hundred episodes per suite informed
+staged checkpoint selection before the final five hundred episode score was
+assembled.
 
-# Installation
+## Model at a glance
 
-Download the repository and create a clean Python 3.12 environment:
+| Property | Released configuration |
+|---|---|
+| Total parameters | 399.084 million |
+| Trainable parameters | 135.107 million |
+| Observation context | Three frames |
+| Predicted action horizon | Seven actions |
+| Executed before replanning | Three actions |
+| Action dimension | Seven |
+| Visual encoder | MAE ViT B |
+| Language encoder | OpenAI CLIP ViT B 32 |
+| Checkpoint format | SafeTensors |
+| Evaluation environment | Headless LIBERO with EGL |
+
+## Run a smoke test
+
+Download the repository and create a clean Python 3.12 environment.
 
 ```bash
 hf download bageldotcom/worlddit --local-dir worlddit
@@ -96,7 +139,7 @@ python -m pip install --no-deps robosuite==1.4.1
 ```
 
 LIBERO supplies the benchmark definitions, assets, and initial states. Keep the
-checkout at `~/LIBERO`, which is the evaluator's default:
+checkout at `~/LIBERO`, which is the evaluator's default.
 
 ```bash
 git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git ~/LIBERO
@@ -105,11 +148,40 @@ git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git ~/LIBERO
 The released evaluation was validated with LIBERO commit
 `8f1084e3132a39270c3a13ebe37270a43ece2a01`.
 
----
+```bash
+python eval.py \
+  --suite libero_spatial \
+  --gpus 1 \
+  --tasks 1 \
+  --episodes 1 \
+  --max-steps 20 \
+  --output-dir results/smoke
+```
+
+A successful smoke test confirms that the environment, checkpoint, visual
+encoders, simulator, and rendering path load together. It is not a benchmark
+result.
 
-# Evaluation
+## How WorldDiT works
 
-## One GPU
+WorldDiT uses three recent observations, robot state, and language as context.
+During training, one diffusion transformer learns a seven step action chunk and
+an auxiliary future visual target. During deployment, the future visual path is
+absent. The policy executes the first three predicted actions, observes again,
+and replans.
+
+> Future visual prediction is a training signal, not a deployment path.
+
+| Training | Deployment |
+|---|---|
+| Action and future visual targets share one backbone | Only the action path remains |
+| Seven action steps are supervised | Seven actions are predicted |
+| Future visual supervision is present | No future visual output is requested |
+| The complete training objective is active | Three actions execute before replanning |
+
+## Evaluation
+
+### One GPU
 
 ```bash
 python eval.py \
@@ -118,7 +190,7 @@ python eval.py \
   --output-dir results/libero_spatial
 ```
 
-## Multiple GPUs
+### Multiple GPUs
 
 ```bash
 CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python eval.py \
@@ -128,11 +200,11 @@ CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python eval.py \
 ```
 
 Each GPU receives an independent progress bar. After all workers finish, rank 0
-prints per-task and overall success rates and writes a structured
+prints per task and overall success rates and writes a structured
 `results.json`. Output directories must be new so an earlier evaluation is
 never overwritten.
 
-Supported suites:
+Supported suites.
 
 ```text
 libero_spatial
@@ -141,21 +213,28 @@ libero_goal
 libero_10
 ```
 
-For a short installation smoke test:
+## What this repository contains
 
-```bash
-python eval.py \
-  --suite libero_spatial \
-  --gpus 1 \
-  --tasks 1 \
-  --episodes 1 \
-  --max-steps 20 \
-  --output-dir results/smoke
+```text
+.
+├── checkpoints/
+│   ├── libero_10/model.safetensors
+│   ├── libero_goal/model.safetensors
+│   ├── libero_object/model.safetensors
+│   └── libero_spatial/model.safetensors
+├── dependencies/
+│   ├── ViT-B-32.pt
+│   └── mae_pretrain_vit_base.pth
+├── eval.py
+├── inference.py
+├── config.json
+└── requirements.txt
 ```
 
----
+`dependencies/` contains the frozen visual and language encoder weights needed
+by the released policy. No additional model downloads are required.
 
-# Inference API
+## Inference API
 
 ```python
 from inference import load_model
@@ -174,9 +253,7 @@ actions = model(primary_images, wrist_images, robot_state, text_tokens)
 
 Evaluation uses the final temporal slot of the predicted action tensor.
 
----
-
-# Architecture Details
+## Architecture details
 
 | Component | Specification |
 |---|---|
@@ -184,20 +261,40 @@ Evaluation uses the final temporal slot of the predicted action tensor.
 | Observation context | 3 frames |
 | Action horizon | 7 actions |
 | Action dimension | 7 |
+| Action aggregation | Temporal ensembling |
 | Language encoder | OpenAI CLIP ViT-B/32 |
 | Visual encoder | MAE ViT-B |
 | Evaluation | Headless LIBERO with EGL |
 | Checkpoint format | SafeTensors |
 
----
+## Intended use
+
+WorldDiT is intended for research on language conditioned robot manipulation in
+the LIBERO simulator. The released checkpoints support reproduction,
+evaluation, and architecture research across the four released suites.
+
+## Scope of the release
+
+The reported results describe LIBERO simulation under the released evaluation
+protocol. They do not establish real robot reliability, safety, or transfer
+across embodiments.
+
+The present release does not isolate the causal contribution of the future
+visual target. Total parameter count also does not measure training cost,
+deployment latency, or runtime efficiency.
+
+## Authors and contact
+
+WorldDiT is developed by Sen Wang, Praveen Rajasekhar, Bidhan Roy, and Marcos
+Villagra at Bagel Labs. Questions can be sent to research@bagel.com.
 
-# Acknowledgments
+## Acknowledgments
 
 This release builds on
 [LIBERO](https://github.com/Lifelong-Robot-Learning/LIBERO),
 [robosuite](https://github.com/ARISE-Initiative/robosuite),
 [OpenAI CLIP](https://github.com/openai/CLIP), and
-[Masked Autoencoders](https://github.com/facebookresearch/mae). Third-party
+[Masked Autoencoders](https://github.com/facebookresearch/mae). Third party
 components remain subject to their respective upstream terms.
 
 ---