Spaces:
Running
Running
Commit Β·
fe1f76f
1
Parent(s): a9ea39e
Update README for the video viewer
Browse filesCo-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
README.md
CHANGED
|
@@ -13,30 +13,21 @@ short_description: Frame-locked viewer for rollouts vs. ground truth
|
|
| 13 |
|
| 14 |
# Rollout vs. ground truth
|
| 15 |
|
| 16 |
-
Side-by-side viewer for forward-dynamics world-model rollouts.
|
| 17 |
-
|
| 18 |
-
to any one dataset.
|
| 19 |
|
| 20 |
-
Accepted forms: `owner/name`, `owner/name@revision`, or a pasted
|
| 21 |
-
|
| 22 |
|
| 23 |
## What it shows
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
- **Rerun** β the same window in a [Rerun](https://rerun.io) viewer: every camera and every
|
| 33 |
-
action channel on one shared frame timeline, so scrubbing moves all of them together. One row
|
| 34 |
-
per camera (GT | generated | optional diff) over a time-series plot of the actions, with the
|
| 35 |
-
critical frame marked. Built only when you open the tab, and only for the cameras selected.
|
| 36 |
-
- **All windows** β sortable table of every window; click a row to load it.
|
| 37 |
-
|
| 38 |
-
Filter by task, gripper edge and arm; sort by critical-frame PSNR to jump to the hardest or
|
| 39 |
-
easiest cases.
|
| 40 |
|
| 41 |
## Expected dataset layout
|
| 42 |
|
|
@@ -47,51 +38,39 @@ metrics_summary.json # optional: [{id, task, edge, arm, full_ms
|
|
| 47 |
# crit_mse, crit_psnr}, ...]
|
| 48 |
windows/<episode_id>/gt.mp4 # required
|
| 49 |
windows/<episode_id>/generated.mp4 # required, same geometry and length as gt.mp4
|
| 50 |
-
windows/<episode_id>/actions.json # optional: [T, D] driving actions
|
| 51 |
-
windows/<episode_id>/meta.json # optional: per-window metadata
|
| 52 |
```
|
| 53 |
|
| 54 |
-
Only the two mp4s are required. With no `windows_manifest.json` the
|
| 55 |
-
listing `windows/` one level deep
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
GT and generated.
|
| 59 |
|
| 60 |
## Storage model
|
| 61 |
|
| 62 |
-
Nothing is downloaded at build time
|
| 63 |
-
|
| 64 |
|
| 65 |
- Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory
|
| 66 |
listing when there is no manifest.
|
| 67 |
-
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
`SESSION_TTL` and directories orphaned by an earlier process, so nothing accumulates if a tab
|
| 74 |
-
dies without a clean shutdown.
|
| 75 |
-
|
| 76 |
-
Rerun recordings follow the same rule: built on first view of the tab, cached beside the
|
| 77 |
-
composed mp4, and evicted with their window. Frames go into the recording as JPEG (quality 90) β
|
| 78 |
-
a raw recording of one 3-camera window is ~16 MB against ~1.5 MB compressed, which is the
|
| 79 |
-
difference between usable and not over a browser connection. The trade-off is that extreme zoom
|
| 80 |
-
in the Rerun viewer shows JPEG artifacts; the **Frame stepper** tab renders from the raw decoded
|
| 81 |
-
frames if you need a pixel-exact look.
|
| 82 |
-
|
| 83 |
-
The sidebar shows live occupancy (`cached this session: n/N windows`).
|
| 84 |
|
| 85 |
## Configuration
|
| 86 |
|
| 87 |
| Variable | Default | Purpose |
|
| 88 |
|---|---|---|
|
| 89 |
-
| `DATASET_REPO` | `OneWorld-AI/abc-critical-window-rollout` | Repo prefilled in the input box. |
|
| 90 |
| `HF_TOKEN` | β | **Required for private datasets.** Set under *Settings β Variables and secrets*. |
|
| 91 |
-
| `MAX_WINDOWS_ON_DISK` | `12` |
|
| 92 |
-
| `MAX_DISCOVER` | `5000` | Cap on
|
| 93 |
| `SESSION_TTL` | `3600` | Seconds of idleness before a session's storage is reclaimed. |
|
| 94 |
|
|
|
|
|
|
|
|
|
|
| 95 |
## Running locally
|
| 96 |
|
| 97 |
```bash
|
|
|
|
| 13 |
|
| 14 |
# Rollout vs. ground truth
|
| 15 |
|
| 16 |
+
Side-by-side viewer for forward-dynamics world-model rollouts. The page starts empty and assumes
|
| 17 |
+
no dataset β type a **dataset repo id** into the box, press **Load**, and pick an episode.
|
|
|
|
| 18 |
|
| 19 |
+
Accepted forms: `owner/name`, `owner/name@revision`, or a pasted
|
| 20 |
+
`https://huggingface.co/datasets/β¦` URL.
|
| 21 |
|
| 22 |
## What it shows
|
| 23 |
|
| 24 |
+
Ground truth and the generated rollout are composed into a **single video**, so the two panels
|
| 25 |
+
stay frame-locked; two independent players drift apart and cannot be scrubbed together. The strip
|
| 26 |
+
carries in-frame captions, a frame counter, and a timeline marking the critical frame.
|
| 27 |
+
|
| 28 |
+
Sidebar controls: search, sort (critical-frame PSNR, task, manifest order), the episode list, a
|
| 29 |
+
camera selector, an optional `|GT β generated|` difference panel, and playback fps β a 33-frame
|
| 30 |
+
window is only ~1 s at native 30 fps. PSNR and MSE for the selected episode are shown underneath.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
## Expected dataset layout
|
| 33 |
|
|
|
|
| 38 |
# crit_mse, crit_psnr}, ...]
|
| 39 |
windows/<episode_id>/gt.mp4 # required
|
| 40 |
windows/<episode_id>/generated.mp4 # required, same geometry and length as gt.mp4
|
|
|
|
|
|
|
| 41 |
```
|
| 42 |
|
| 43 |
+
Only the two mp4s are required. With no `windows_manifest.json` the episode list is discovered by
|
| 44 |
+
listing `windows/` one level deep. Videos may stack **N cameras vertically** β N is inferred from
|
| 45 |
+
the frame aspect ratio and the camera selector adapts. Black letterbox bars are trimmed
|
| 46 |
+
automatically, identically for both videos.
|
|
|
|
| 47 |
|
| 48 |
## Storage model
|
| 49 |
|
| 50 |
+
Nothing is downloaded at build time and no repo is ever pulled whole, so datasets far larger than
|
| 51 |
+
the Space still work.
|
| 52 |
|
| 53 |
- Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory
|
| 54 |
listing when there is no manifest.
|
| 55 |
+
- An episode's mp4s are fetched the first time that episode is opened, never in bulk.
|
| 56 |
+
- Each browser session gets its own scratch directory. At most `MAX_WINDOWS_ON_DISK` episodes stay
|
| 57 |
+
resident; beyond that the least recently used are evicted along with their rendered videos.
|
| 58 |
+
Decoded frames are capped at 2 episodes in RAM and composed strips at 1.
|
| 59 |
+
- The directory is deleted when the session ends. A janitor also reaps sessions idle past
|
| 60 |
+
`SESSION_TTL` and directories orphaned by an earlier process.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
## Configuration
|
| 63 |
|
| 64 |
| Variable | Default | Purpose |
|
| 65 |
|---|---|---|
|
|
|
|
| 66 |
| `HF_TOKEN` | β | **Required for private datasets.** Set under *Settings β Variables and secrets*. |
|
| 67 |
+
| `MAX_WINDOWS_ON_DISK` | `12` | Episodes kept per session before LRU eviction. |
|
| 68 |
+
| `MAX_DISCOVER` | `5000` | Cap on episodes listed when a repo has no manifest. |
|
| 69 |
| `SESSION_TTL` | `3600` | Seconds of idleness before a session's storage is reclaimed. |
|
| 70 |
|
| 71 |
+
Note: on a public Space the `HF_TOKEN` is used to read whatever repo a visitor types, so scope it
|
| 72 |
+
to only the datasets you are willing to expose.
|
| 73 |
+
|
| 74 |
## Running locally
|
| 75 |
|
| 76 |
```bash
|