typoverflow Claude Opus 5 (1M context) commited on
Commit
fe1f76f
Β·
1 Parent(s): a9ea39e

Update README for the video viewer

Browse files

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +28 -49
README.md CHANGED
@@ -13,30 +13,21 @@ short_description: Frame-locked viewer for rollouts vs. ground truth
13
 
14
  # Rollout vs. ground truth
15
 
16
- Side-by-side viewer for forward-dynamics world-model rollouts. Type a **dataset repo id** into
17
- the box at the top and it renders that repo β€” the Space holds no data of its own and is not tied
18
- to any one dataset.
19
 
20
- Accepted forms: `owner/name`, `owner/name@revision`, or a pasted `https://huggingface.co/datasets/…`
21
- URL. `DATASET_REPO` sets what is prefilled at startup.
22
 
23
  ## What it shows
24
 
25
- - **Playback** β€” GT and generated rendered into one video, so the two panels stay frame-locked
26
- instead of drifting apart the way two independent players do. Optional `|GT βˆ’ generated|`
27
- difference panel. Playback fps is adjustable (a 33-frame window is only ~1 s at 30 fps).
28
- - **Frame stepper** β€” a single frame from both videos side by side, defaulting to the critical
29
- frame when the dataset marks one.
30
- - **Driving actions** β€” the action chunk fed to the model. A 14-D chunk is split into grippers /
31
- left-arm / right-arm; any other width is plotted as raw channels.
32
- - **Rerun** β€” the same window in a [Rerun](https://rerun.io) viewer: every camera and every
33
- action channel on one shared frame timeline, so scrubbing moves all of them together. One row
34
- per camera (GT | generated | optional diff) over a time-series plot of the actions, with the
35
- critical frame marked. Built only when you open the tab, and only for the cameras selected.
36
- - **All windows** β€” sortable table of every window; click a row to load it.
37
-
38
- Filter by task, gripper edge and arm; sort by critical-frame PSNR to jump to the hardest or
39
- easiest cases.
40
 
41
  ## Expected dataset layout
42
 
@@ -47,51 +38,39 @@ metrics_summary.json # optional: [{id, task, edge, arm, full_ms
47
  # crit_mse, crit_psnr}, ...]
48
  windows/<episode_id>/gt.mp4 # required
49
  windows/<episode_id>/generated.mp4 # required, same geometry and length as gt.mp4
50
- windows/<episode_id>/actions.json # optional: [T, D] driving actions
51
- windows/<episode_id>/meta.json # optional: per-window metadata
52
  ```
53
 
54
- Only the two mp4s are required. With no `windows_manifest.json` the window list is discovered by
55
- listing `windows/` one level deep, and per-window metadata is read from each `meta.json` as you
56
- open it. Videos may stack **N cameras vertically** β€” N is inferred from the frame aspect ratio,
57
- and the camera selector adapts. Black letterbox bars are trimmed automatically, identically for
58
- GT and generated.
59
 
60
  ## Storage model
61
 
62
- Nothing is downloaded at build time, and no repo is ever pulled whole β€” important for datasets
63
- too large to fit on the Space.
64
 
65
  - Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory
66
  listing when there is no manifest.
67
- - A window's mp4s, `actions.json` and `meta.json` are fetched **the first time that window is
68
- opened**, never in bulk.
69
- - Each browser session gets its own scratch directory. At most `MAX_WINDOWS_ON_DISK` windows stay
70
- resident; beyond that the least recently used are deleted. Decoded frames are capped at 2
71
- windows in RAM and composed strips at 1.
72
- - The whole directory is deleted when the session ends. A janitor also reaps sessions idle past
73
- `SESSION_TTL` and directories orphaned by an earlier process, so nothing accumulates if a tab
74
- dies without a clean shutdown.
75
-
76
- Rerun recordings follow the same rule: built on first view of the tab, cached beside the
77
- composed mp4, and evicted with their window. Frames go into the recording as JPEG (quality 90) β€”
78
- a raw recording of one 3-camera window is ~16 MB against ~1.5 MB compressed, which is the
79
- difference between usable and not over a browser connection. The trade-off is that extreme zoom
80
- in the Rerun viewer shows JPEG artifacts; the **Frame stepper** tab renders from the raw decoded
81
- frames if you need a pixel-exact look.
82
-
83
- The sidebar shows live occupancy (`cached this session: n/N windows`).
84
 
85
  ## Configuration
86
 
87
  | Variable | Default | Purpose |
88
  |---|---|---|
89
- | `DATASET_REPO` | `OneWorld-AI/abc-critical-window-rollout` | Repo prefilled in the input box. |
90
  | `HF_TOKEN` | β€” | **Required for private datasets.** Set under *Settings β†’ Variables and secrets*. |
91
- | `MAX_WINDOWS_ON_DISK` | `12` | Windows kept per session before LRU eviction. |
92
- | `MAX_DISCOVER` | `5000` | Cap on windows listed when a repo has no manifest. |
93
  | `SESSION_TTL` | `3600` | Seconds of idleness before a session's storage is reclaimed. |
94
 
 
 
 
95
  ## Running locally
96
 
97
  ```bash
 
13
 
14
  # Rollout vs. ground truth
15
 
16
+ Side-by-side viewer for forward-dynamics world-model rollouts. The page starts empty and assumes
17
+ no dataset β€” type a **dataset repo id** into the box, press **Load**, and pick an episode.
 
18
 
19
+ Accepted forms: `owner/name`, `owner/name@revision`, or a pasted
20
+ `https://huggingface.co/datasets/…` URL.
21
 
22
  ## What it shows
23
 
24
+ Ground truth and the generated rollout are composed into a **single video**, so the two panels
25
+ stay frame-locked; two independent players drift apart and cannot be scrubbed together. The strip
26
+ carries in-frame captions, a frame counter, and a timeline marking the critical frame.
27
+
28
+ Sidebar controls: search, sort (critical-frame PSNR, task, manifest order), the episode list, a
29
+ camera selector, an optional `|GT βˆ’ generated|` difference panel, and playback fps β€” a 33-frame
30
+ window is only ~1 s at native 30 fps. PSNR and MSE for the selected episode are shown underneath.
 
 
 
 
 
 
 
 
31
 
32
  ## Expected dataset layout
33
 
 
38
  # crit_mse, crit_psnr}, ...]
39
  windows/<episode_id>/gt.mp4 # required
40
  windows/<episode_id>/generated.mp4 # required, same geometry and length as gt.mp4
 
 
41
  ```
42
 
43
+ Only the two mp4s are required. With no `windows_manifest.json` the episode list is discovered by
44
+ listing `windows/` one level deep. Videos may stack **N cameras vertically** β€” N is inferred from
45
+ the frame aspect ratio and the camera selector adapts. Black letterbox bars are trimmed
46
+ automatically, identically for both videos.
 
47
 
48
  ## Storage model
49
 
50
+ Nothing is downloaded at build time and no repo is ever pulled whole, so datasets far larger than
51
+ the Space still work.
52
 
53
  - Loading a repo fetches only the manifest and metrics JSONs (a few hundred KB), or one directory
54
  listing when there is no manifest.
55
+ - An episode's mp4s are fetched the first time that episode is opened, never in bulk.
56
+ - Each browser session gets its own scratch directory. At most `MAX_WINDOWS_ON_DISK` episodes stay
57
+ resident; beyond that the least recently used are evicted along with their rendered videos.
58
+ Decoded frames are capped at 2 episodes in RAM and composed strips at 1.
59
+ - The directory is deleted when the session ends. A janitor also reaps sessions idle past
60
+ `SESSION_TTL` and directories orphaned by an earlier process.
 
 
 
 
 
 
 
 
 
 
 
61
 
62
  ## Configuration
63
 
64
  | Variable | Default | Purpose |
65
  |---|---|---|
 
66
  | `HF_TOKEN` | β€” | **Required for private datasets.** Set under *Settings β†’ Variables and secrets*. |
67
+ | `MAX_WINDOWS_ON_DISK` | `12` | Episodes kept per session before LRU eviction. |
68
+ | `MAX_DISCOVER` | `5000` | Cap on episodes listed when a repo has no manifest. |
69
  | `SESSION_TTL` | `3600` | Seconds of idleness before a session's storage is reclaimed. |
70
 
71
+ Note: on a public Space the `HF_TOKEN` is used to read whatever repo a visitor types, so scope it
72
+ to only the datasets you are willing to expose.
73
+
74
  ## Running locally
75
 
76
  ```bash