moge_depth/README.md

MoGe-2 monocular depth — qualitative previews

Full-length monocular depth over the clips published in this bucket, rendered as a three-panel video: RGB | depth | normal, each panel at the clip's native 1280x720.

These are previews for judging detail by eye. They are not a dataset tier and nothing in the pipeline consumes them — the pipeline's own depth comes from PointWorld scene_flows (sparse metric 3D tracks), not from MoGe.

Model

Ruicheng/moge-2-vitl-normal, resolution_level=9.

resolution_level=9 is the default and already maps to the top of MoGe-2's num_tokens_range (3600 tokens) — the finest setting the model offers.

MoGe-3 is not used, because it does not exist as runnable code. The paper (arXiv 2607.17967, "Fine-Detail Monocular Geometry Estimation…") was announced in the upstream README on 2026-07-21, but at pinned commit 925b8ed — still origin/main as of this run — there is no moge/model/v3.py and the pretrained table lists moge-3-vitg/moge-3-vitl as "coming soon" with no HF repo id.

How to read the panels

• Depth uses one global normalisation for the whole clip (robust 2/98 percentiles over every valid pixel of every frame), never per frame. Per-frame normalisation makes the panel breathe as the near/far extremes move, which by eye is indistinguishable from the model's own temporal instability — the exact thing these videos exist to let someone judge. The colour range in metres is burned into the panel label.
• Normal is where fine detail actually shows. Depth is dominated by the metre-scale front-to-back ramp of the workbench; the normal map spends its whole colour budget on surface relief instead.
• No fov_x was passed, so MoGe estimates its own FOV per frame even though this project knows the true DROID intrinsics. That is the out-of-the-box path — priming the model with the calibration would measure a best case that arbitrary video would not get.
• Videos are muxed at 15 fps, the DROID trajectory rate. The source mp4 containers advertise 60/1, but their frames are the 15 Hz trajectory samples (this repo indexes poses per video frame at DEFAULT_TRAJECTORY_FPS = 15.0). Playing at 60 would look like temporal jitter that is not there.

Clips

file	camera	frames	depth p2–p98 (m)	size
AUTOLab+0d4edc83+2023-10-21-19h-06m-10s__22008760.mp4	ext1	178	0.31–1.41	20.7 MB
AUTOLab+0d4edc83+2023-10-21-19h-07m-04s__22008760.mp4	ext1	127	0.27–1.48	15.3 MB
AUTOLab+0d4edc83+2023-10-21-19h-07m-04s__24400334.mp4	ext2	127	0.42–1.65	15.5 MB
AUTOLab+0d4edc83+2023-10-21-19h-07m-18s__22008760.mp4	ext1	146	0.25–1.46	16.9 MB
AUTOLab+0d4edc83+2023-10-21-19h-08m-08s__22008760.mp4	ext1	148	0.26–1.48	17.7 MB
AUTOLab+0d4edc83+2023-10-21-19h-08m-25s__22008760.mp4	ext1	133	0.27–1.45	16.0 MB
AUTOLab+0d4edc83+2023-10-21-19h-08m-41s__22008760.mp4	ext1	165	0.28–1.46	18.8 MB
AUTOLab+0d4edc83+2023-10-21-19h-08m-58s__22008760.mp4	ext1	198	0.19–1.36	23.0 MB
AUTOLab+0d4edc83+2023-10-21-19h-09m-18s__22008760.mp4	ext1	124	0.27–1.42	15.0 MB
AUTOLab+0d4edc83+2023-10-21-19h-09m-53s__22008760.mp4	ext1	175	0.29–1.49	19.3 MB

All of these are the same scene (AUTOLab+0d4edc83, "Put brick in drawer shelf and close drawer"), recorded seconds apart, because that is the whole of what this bucket holds. Ten clips agreeing therefore is not evidence that MoGe-2 is stable — it is one condition measured ten times. The only genuinely different viewpoint in the set is camera 24400334 (ext2).

object_step.json — the quantitative companion

Whether the model separates the manipulated object from the surface under it, measured with the pipeline's own object mask (master/prompt_0_masks.h5):

• step_mm — mean depth of a 25 px ring around the object minus mean depth inside it. Positive = object in front of its surroundings.
• edge_ratio — median |∇depth| on the mask boundary over the median in the ring. 1.0 = boundary indistinguishable from flat surface. This is the half of the question a step size cannot answer: a model that smears a real 20 mm step over 30 px still reports 20 mm.

Both are computed within a single frame on purpose. With no fov_x given, MoGe estimates its own scale per frame and that scale drifts, so absolute metres are not comparable across frames; an inside-minus-outside difference cancels a per-frame scale factor to first order and is.

Two caveats that travel with these numbers:

• step_mm_p10 goes negative on several clips. That is the measurement's limit, not the model's error: once the gripper closes on the object, the 25 px ring contains the fingers, which are nearer the camera than the object, so the difference flips sign. The median is robust to this; the p10 is not.
• Two clips have no prompt_0_masks.h5 at all (...07m-04s/24400334, ...09m-53s/22008760) — both are the entries with n_win=0 in the dataset manifest, i.e. datagen never produced object masks for them. They ship a video and no step.