| --- |
| language: |
| - en |
| library_name: lerobot |
| base_model: nvidia/GR00T-N1.7-3B |
| datasets: |
| - justintiensmith/VLA_Reasoning_Training_Dataset_1200_Trimmed_Start_5_Frame |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - vision-language-action |
| - imitation-learning |
| - so-101 |
| - spa-bench |
| - groot-n1.7 |
| license: apache-2.0 |
| --- |
| |
| # GR00T-N1.7 Frozen LLM — Spa-Bench Epoch 12 |
|
|
| This is the GR00T-N1.7 checkpoint evaluated as the **Frozen LLM** condition in |
| Spa-Bench. “Frozen LLM” means that the language-model parameters were held |
| fixed; the policy continued to receive language instructions. |
|
|
| ## Model details |
|
|
| | Field | Value | |
| | --- | --- | |
| | Model repository | [`justintiensmith/groot_multi_gpu_v4`](https://huggingface.co/justintiensmith/groot_multi_gpu_v4) | |
| | Base model | [`nvidia/GR00T-N1.7-3B`](https://huggingface.co/nvidia/GR00T-N1.7-3B) | |
| | Checkpoint | End of epoch 12; step 71,304 | |
| | Robot | SO-101 single-arm manipulator | |
| | Inputs | Fixed middle RGB, wrist RGB, six absolute joint positions, text instruction | |
| | Outputs | Six absolute joint-position targets | |
| | Action horizon | 16 | |
| | Adaptation | LLM frozen; visual, multimodal/projector, VLLN, and diffusion-action modules updated | |
| | Optimizer | AdamW, learning rate `1e-4`, weight decay `1e-5` | |
| | Schedule | 5% warm-up, then cosine decay | |
| | Hardware and batch | Four NVIDIA GH200 GPUs; 24 samples per device, global batch 96 | |
|
|
| The embedded `train_config.json` pins the motion-trimmed training data to |
| [`justintiensmith/VLA_Reasoning_Training_Dataset_1200_Trimmed_Start_5_Frame@c1231a8`](https://huggingface.co/datasets/justintiensmith/VLA_Reasoning_Training_Dataset_1200_Trimmed_Start_5_Frame/tree/c1231a8f2b282b3f875fc898dced9ebf5574b903). |
| That exact snapshot is a two-camera projection of the repository's five-camera |
| trimmed release and is preserved by the `groot-frozen-training-snapshot` tag. |
| The tabular trajectory file and middle/wrist videos are byte-identical between |
| the two snapshots; episode metadata differs because the unused camera keys were |
| removed. Both contain 1,200 episodes, 570,386 frames, and 321 instruction |
| strings. |
|
|
| The run records the base-model identifier but not its immutable source |
| revision. The current public GR00T base revision at archival review is |
| `2fc962b973bccdd5d8ce4f67cc63b264d6886495`; it must not be assumed to be the |
| unrecorded training revision. |
|
|
| The author-supplied original |
| [`train_groot_n17_frozen_llm.sh`](https://github.com/justintiensmith/Imperial-Thesis/blob/main/training/groot/train_groot_n17_frozen_llm.sh) |
| launcher is archived with the thesis artifact. It records seed 42, four-GPU |
| training, the immutable dataset revision, component-freezing flags, checkpoint |
| cadence, and the historical environment paths used for this run. |
|
|
| ## Deployment processing |
|
|
| For both GR00T variants, non-gripper action dimensions used the causal filter |
| `filtered = 0.25 × current + 0.75 × previous_filtered`. Filter state was |
| initialized from the measured robot state and reset for every rollout. The |
| gripper command was excluded. |
|
|
| ## Physical evaluation |
|
|
| | Condition | Successes | Rate | |
| | --- | ---: | ---: | |
| | Familiar/in-distribution spatial instructions | 81/120 | 67.5% | |
| | All withheld/OOD spatial configurations | 133/300 | 44.3% | |
| | Matched OOD subset | 50/120 | 41.7% | |
| | Matched direct-manipulation controls | 98/120 | 81.7% | |
|
|
| The matched-control gap was 40.0 percentage points. These are physical rollout |
| results, not simulation metrics. |
|
|
| - Rollouts: [`justintiensmith/Spa_Bench_Full_GR00T-N1.7_Frozen_LLM`](https://huggingface.co/datasets/justintiensmith/Spa_Bench_Full_GR00T-N1.7_Frozen_LLM) |
| - Thesis artifact: [`justintiensmith/Imperial-Thesis`](https://github.com/justintiensmith/Imperial-Thesis) |
|
|
| ## Intended use and limitations |
|
|
| This release supports reproduction and analysis of the Spa-Bench experiment. |
| Older project files may call it “vision-only”; that term is inaccurate because |
| several non-language modules were updated. The motion-trimmed data and |
| deployment smoothing are policy-specific choices, so this comparison is not a |
| controlled architecture ablation. |
|
|
| Robot policies can move hardware unexpectedly. Use conservative motion limits, |
| an accessible emergency stop, a clear workspace, and direct supervision. Do |
| not deploy this checkpoint for unattended or safety-critical operation. |
|
|
| ## Citation |
|
|
| Please cite the [completed Spa-Bench MSc report](https://github.com/justintiensmith/Imperial-Thesis/blob/main/thesis/report/spa-bench-msc-thesis.pdf), |
| the [thesis artifact](https://github.com/justintiensmith/Imperial-Thesis), and |
| the GR00T-N1.7 work referenced in the report. |
|
|
