bagellabs commited on
Commit
4b7fb95
verified
1 Parent(s): c0548c1

Improve the WorldDiT model card layout

Browse files

Reorganizes the WorldDiT card into a compact release dashboard. The rollout showcase now uses a two by two layout, the results and released configuration share one snapshot, and advanced reference material is grouped into expandable sections. Model files, checkpoints, commands, results, and metadata remain unchanged.

Files changed (1) hide show
  1. README.md +77 -82
README.md CHANGED
@@ -38,91 +38,81 @@ widget:
38
  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
39
  </p>
40
 
41
- # WorldDiT
42
 
43
- ## One diffusion backbone learns what to do and what comes next.
44
 
45
  <p align="center">
46
- <a href="https://huggingface.co/bageldotcom/worlddit" target="_blank">
47
- <img src="https://img.shields.io/badge/馃_DOWNLOAD_WORLDDIT_WEIGHTS-FFD21E?style=for-the-badge&logoColor=000000" alt="Download WorldDiT Weights">
48
- </a>
49
- <a href="https://github.com/Lifelong-Robot-Learning/LIBERO" target="_blank">
50
- <img src="https://img.shields.io/badge/馃_LIBERO_BENCHMARK-FF6B6B?style=for-the-badge&logoColor=white" alt="LIBERO Benchmark">
51
- </a>
52
- </p>
53
-
54
  WorldDiT learns continuous robot action chunks and a future visual target
55
  through one shared diffusion transformer. Deployment keeps only the action
56
  path.
 
57
 
 
58
  This release includes four LIBERO checkpoints, a self contained inference
59
  runtime, and an evaluator for reproducing the reported suite results.
 
60
 
61
  ## See WorldDiT act
62
 
63
  The four clips below show successful rollouts from the released checkpoints.
64
  Each clip covers a different LIBERO suite and camera view.
65
 
66
- <Gallery />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
- | Suite | View | Video |
69
  |---|---|---|
70
- | LIBERO Spatial | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4) |
71
- | LIBERO Object | Agent view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4) |
72
- | LIBERO Goal | Side view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4) |
73
- | LIBERO Long | Front view | [Open MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4) |
74
 
75
- ## What is in this release
76
-
77
- | Release component | Included artifact |
78
- |---|---|
79
- | LIBERO Spatial policy | SafeTensors checkpoint |
80
- | LIBERO Object policy | SafeTensors checkpoint |
81
- | LIBERO Goal policy | SafeTensors checkpoint |
82
- | LIBERO Long policy | SafeTensors checkpoint |
83
- | Model runtime | `inference.py` |
84
- | Evaluation runtime | `eval.py` |
85
- | Frozen encoders | CLIP ViT B 32 and MAE ViT B |
86
- | Configuration | `config.json` |
87
- | Environment | Pinned Python requirements |
88
-
89
- The repository is self contained for WorldDiT inference. LIBERO still provides
90
- the benchmark environments, assets, task definitions, and initial states.
91
-
92
- ## Reported LIBERO results
93
-
94
- Across the four released suite checkpoints, WorldDiT records 1,898 successful
95
- episodes out of 2,000 under the selection aware evaluation protocol.
96
-
97
- | Suite | Successful episodes | Success rate |
98
- |---|---:|---:|
99
- | LIBERO Spatial | 490 of 500 | 98.0 percent |
100
- | LIBERO Object | 485 of 500 | 97.0 percent |
101
- | LIBERO Goal | 464 of 500 | 92.8 percent |
102
- | LIBERO Long | 459 of 500 | 91.8 percent |
103
- | Selection aware mean | 1,898 of 2,000 | 94.9 percent |
104
 
105
  The released runtime and checkpoints were revalidated from a clean installation
106
- on eight RTX Pro 6000 Blackwell GPUs.
107
-
108
- The result is selection aware because three hundred episodes per suite informed
109
- staged checkpoint selection before the final five hundred episode score was
110
- assembled.
111
-
112
- ## Model at a glance
113
-
114
- | Property | Released configuration |
115
- |---|---|
116
- | Total parameters | 399.084 million |
117
- | Trainable parameters | 135.107 million |
118
- | Observation context | Three frames |
119
- | Predicted action horizon | Seven actions |
120
- | Executed before replanning | Three actions |
121
- | Action dimension | Seven |
122
- | Visual encoder | MAE ViT B |
123
- | Language encoder | OpenAI CLIP ViT B 32 |
124
- | Checkpoint format | SafeTensors |
125
- | Evaluation environment | Headless LIBERO with EGL |
126
 
127
  ## Run a smoke test
128
 
@@ -179,7 +169,10 @@ and replans.
179
  | Future visual supervision is present | No future visual output is requested |
180
  | The complete training objective is active | Three actions execute before replanning |
181
 
182
- ## Evaluation
 
 
 
183
 
184
  ### One GPU
185
 
@@ -213,7 +206,10 @@ libero_goal
213
  libero_10
214
  ```
215
 
216
- ## What this repository contains
 
 
 
217
 
218
  ```text
219
  .
@@ -234,7 +230,10 @@ libero_10
234
  `dependencies/` contains the frozen visual and language encoder weights needed
235
  by the released policy. No additional model downloads are required.
236
 
237
- ## Inference API
 
 
 
238
 
239
  ```python
240
  from inference import load_model
@@ -253,7 +252,10 @@ actions = model(primary_images, wrist_images, robot_state, text_tokens)
253
 
254
  Evaluation uses the final temporal slot of the predicted action tensor.
255
 
256
- ## Architecture details
 
 
 
257
 
258
  | Component | Specification |
259
  |---|---|
@@ -267,21 +269,14 @@ Evaluation uses the final temporal slot of the predicted action tensor.
267
  | Evaluation | Headless LIBERO with EGL |
268
  | Checkpoint format | SafeTensors |
269
 
270
- ## Intended use
271
-
272
- WorldDiT is intended for research on language conditioned robot manipulation in
273
- the LIBERO simulator. The released checkpoints support reproduction,
274
- evaluation, and architecture research across the four released suites.
275
 
276
- ## Scope of the release
277
 
278
- The reported results describe LIBERO simulation under the released evaluation
279
- protocol. They do not establish real robot reliability, safety, or transfer
280
- across embodiments.
281
-
282
- The present release does not isolate the causal contribution of the future
283
- visual target. Total parameter count also does not measure training cost,
284
- deployment latency, or runtime efficiency.
285
 
286
  ## Authors and contact
287
 
 
38
  <img src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/paris2/model-card/v1/bagel_labs_logo.png" alt="Bagel Labs">
39
  </p>
40
 
41
+ <h1 align="center">WorldDiT</h1>
42
 
43
+ <h2 align="center">One diffusion backbone learns what to do and what comes next.</h2>
44
 
45
  <p align="center">
 
 
 
 
 
 
 
 
46
  WorldDiT learns continuous robot action chunks and a future visual target
47
  through one shared diffusion transformer. Deployment keeps only the action
48
  path.
49
+ </p>
50
 
51
+ <p align="center">
52
  This release includes four LIBERO checkpoints, a self contained inference
53
  runtime, and an evaluator for reproducing the reported suite results.
54
+ </p>
55
 
56
  ## See WorldDiT act
57
 
58
  The four clips below show successful rollouts from the released checkpoints.
59
  Each clip covers a different LIBERO suite and camera view.
60
 
61
+ <table>
62
+ <tr>
63
+ <td width="50%" valign="top">
64
+ <video width="100%" controls muted loop playsinline preload="metadata"
65
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4"></video>
66
+ <strong>LIBERO Spatial</strong><br>
67
+ Task 5, front view.
68
+ </td>
69
+ <td width="50%" valign="top">
70
+ <video width="100%" controls muted loop playsinline preload="metadata"
71
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4"></video>
72
+ <strong>LIBERO Object</strong><br>
73
+ Task 8, agent view.
74
+ </td>
75
+ </tr>
76
+ <tr>
77
+ <td width="50%" valign="top">
78
+ <video width="100%" controls muted loop playsinline preload="metadata"
79
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4"></video>
80
+ <strong>LIBERO Goal</strong><br>
81
+ Task 10, side view.
82
+ </td>
83
+ <td width="50%" valign="top">
84
+ <video width="100%" controls muted loop playsinline preload="metadata"
85
+ src="https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4"></video>
86
+ <strong>LIBERO Long</strong><br>
87
+ Task 6, front view.
88
+ </td>
89
+ </tr>
90
+ </table>
91
+
92
+ [Spatial MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_spatial_frontview_task05_episode01.mp4)
93
+ 路 [Object MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_object_agentview_task08_episode01.mp4)
94
+ 路 [Goal MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_goal_sideview_task10_episode01.mp4)
95
+ 路 [Long MP4](https://pub-2c09ae97630f4932a23e622b450076e0.r2.dev/worlddit/model-card/v1/worlddit_libero_10_frontview_task06_episode01.mp4)
96
+
97
+ ## Release snapshot
98
+
99
+ | Reported LIBERO result | Released model |
100
+ |---|---|
101
+ | **94.9 percent** selection aware mean<br>**1,898 of 2,000** successful episodes | **399.084 million** total parameters<br>**135.107 million** trainable parameters |
102
+ | **98.0 percent** Spatial<br>**97.0 percent** Object | **Three** observation frames<br>**Seven** predicted actions |
103
+ | **92.8 percent** Goal<br>**91.8 percent** Long | **Three** actions executed before replanning<br>**Seven** action dimensions |
104
 
105
+ | Checkpoints | Runtime | Encoders and environment |
106
  |---|---|---|
107
+ | Spatial<br>Object<br>Goal<br>Long | `inference.py`<br>`eval.py`<br>`config.json` | MAE ViT B<br>OpenAI CLIP ViT B 32<br>SafeTensors and pinned requirements |
 
 
 
108
 
109
+ The repository is self contained for WorldDiT inference. LIBERO provides the
110
+ benchmark environments, assets, task definitions, and initial states.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
 
112
  The released runtime and checkpoints were revalidated from a clean installation
113
+ on eight RTX Pro 6000 Blackwell GPUs. The result is selection aware because
114
+ three hundred episodes per suite informed staged checkpoint selection before
115
+ the final five hundred episode score was assembled.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
  ## Run a smoke test
118
 
 
169
  | Future visual supervision is present | No future visual output is requested |
170
  | The complete training objective is active | Three actions execute before replanning |
171
 
172
+ ## Reference
173
+
174
+ <details>
175
+ <summary><strong>Full evaluation commands</strong></summary>
176
 
177
  ### One GPU
178
 
 
206
  libero_10
207
  ```
208
 
209
+ </details>
210
+
211
+ <details>
212
+ <summary><strong>Repository contents</strong></summary>
213
 
214
  ```text
215
  .
 
230
  `dependencies/` contains the frozen visual and language encoder weights needed
231
  by the released policy. No additional model downloads are required.
232
 
233
+ </details>
234
+
235
+ <details>
236
+ <summary><strong>Inference API and tensor shapes</strong></summary>
237
 
238
  ```python
239
  from inference import load_model
 
252
 
253
  Evaluation uses the final temporal slot of the predicted action tensor.
254
 
255
+ </details>
256
+
257
+ <details>
258
+ <summary><strong>Architecture details</strong></summary>
259
 
260
  | Component | Specification |
261
  |---|---|
 
269
  | Evaluation | Headless LIBERO with EGL |
270
  | Checkpoint format | SafeTensors |
271
 
272
+ </details>
 
 
 
 
273
 
274
+ ## Use and scope
275
 
276
+ | Intended use | Scope of the release |
277
+ |---|---|
278
+ | Research on language conditioned robot manipulation in the LIBERO simulator. The released checkpoints support reproduction, evaluation, and architecture research across the four released suites. | The results describe LIBERO simulation under the released evaluation protocol. They do not establish real robot reliability, safety, or transfer across embodiments. |
279
+ | The released checkpoints cover all four LIBERO suites. | The release does not isolate the causal contribution of the future visual target. Total parameter count does not measure training cost, deployment latency, or runtime efficiency. |
 
 
 
280
 
281
  ## Authors and contact
282