AlexHaochenLi/scannet_selfloc

WhereAmI: ScanNet self-localization data

This directory is a standalone generator for training a model to infer its camera cell and one of eight display-grid facing vectors from an ASCII object grid and a multi-image ScanNet RGB input. It does not import the VSI-590K QA-construction code; the needed visibility, blur, and coherent-clip routines are local in common.py.

Each JSONL record has a hidden target camera cell/facing vector, a cropped ASCII object grid, the RGB image paths, and audit metadata. The camera is never drawn into the grid. Multi-image groups are deliberately expanded into separate records, one for every eligible anchor image; image_group_id and anchor_image_index retain this correlation.

Labels and filters

• The target uses CAMERA_CELL: (x, y) and FACING_VECTOR: [dx, dy].
• Facing labels are limited to [0, -1], [1, -1], [1, 0], [1, 1], [0, 1], [-1, 1], [-1, 0], and [-1, -1].
• Blur filtering uses Laplacian variance with the original pipeline defaults: at least 50.0 and the per-scene 30th-percentile score.
• Labels close to an eight-way angular boundary are dropped. The default keeps only labels at least 5 degrees from a 22.5-degree boundary.
• Camera centers must lie in the central 0.9 m x 0.9 m square of their 1 m grid cell by default: each x/z offset from the cell center must be at most 0.45 m.
• Visible object categories and their intersection with the object grid are retained as audit metadata, but they do not determine anchor eligibility.
• Multi-image groups draw their candidates from fixed 180-frame sliding windows. There is no hard visible-instance-overlap or persistent-landmark requirement: those annotations are incomplete and cannot reliably prove two images lack shared visual evidence. A group's inclusive source_span runs from its first to its last selected keyframe; once accepted, that span is unavailable to every later group from the same scene, preventing overlap across trajectory segments.
• Groups are greedily selected from anchor-eligible frames. Starting from the earliest frame in a window, each later image must differ from the immediately previous selected image by at least 0.50 m of camera motion or 20 degrees of yaw. A group therefore contains between two and --max-multi-images images without requiring a minimum frame-index gap.
• generate_scannet_self_localization_uniform.py builds baseline groups by splitting the video into --uniform-window-size frame windows and uniformly sampling --uniform-frames-per-window frames in each window. Each sampled slot reuses the same blur, cell-center, and facing-boundary eligibility checks; if the sampled frame is not eligible, the generator searches within --uniform-replacement-radius frames by nearest distance, breaking ties to the earlier frame, and avoids reusing selected frames within the scene. If no eligible replacement exists in that range, the slot is kept as an input image but does not generate an anchor question.

Small local smoke run

/home/haochen/anaconda3/envs/vsibench/bin/python \
  whereami/generate_scannet_self_localization.py \
  --scene-id scene0295_00 \
  --max-multi-groups-per-scene 1 \
  --output-jsonl whereami/results/debug_selfloc.jsonl \
  --summary-json whereami/results/debug_selfloc_generation_summary.json \
  --media-root whereami/results/debug_selfloc_images

/home/haochen/anaconda3/envs/vsibench/bin/python \
  whereami/summarize_self_localization.py \
  --input-jsonl whereami/results/debug_selfloc.jsonl

/home/haochen/anaconda3/envs/vsibench/bin/python \
  whereami/visualize_self_localization_samples.py \
  --input-jsonl whereami/results/debug_selfloc.jsonl \
  --output-html whereami/results/debug_selfloc.html \
  --examples 8

Uniform baseline smoke run:

/home/haochen/anaconda3/envs/vsibench/bin/python \
  whereami/generate_scannet_self_localization_uniform.py \
  --scene-id scene0295_00 \
  --output-jsonl whereami/results/debug_selfloc_uniform_w600_s12.jsonl \
  --summary-json whereami/results/debug_selfloc_uniform_w600_s12_generation_summary.json \
  --media-root whereami/results/debug_selfloc_uniform_w600_s12_images

For the other server, point --vsi-root, --data-root, --video-root, and --grid-root at that server's 200-scene layout. The manifest's historical absolute paths are rebased to these roots when they are unavailable.