# GR00T Whole-Body Control Real-World Benchmark This example reports a real-world Unitree G1 evaluation using **Isaac GR00T N1.7** together with **GR00T Whole-Body Control / GEAR-SONIC**. The benchmark focuses on everyday mobile-manipulation tasks that require walking, table approach, grasping, foot placement, and whole-body pickup motions. ## Summary GR00T N1.7 with SONIC can execute closed-loop whole-body skills on a real humanoid robot. The main targeted task is **walk to a table and pick up an object**. A single mixed-object policy was trained from demonstrations covering about 50 table-top objects, then evaluated systematically across 8 representative objects in the training set. This follows the object-pickup setting studied in the [SONIC paper](https://nvlabs.github.io/GEAR-SONIC/static/pdf/sonic_paper.pdf), where the robot walks to a table, locates a target object, and grasps it under randomized table heights and object positions. A second task-specific policy was trained for walking to a small table, picking up a soda can, stepping on a trash-can trigger, and dropping the can inside. For more GR00T Whole-Body Control task examples, see the [GEAR-SONIC project page](https://nvlabs.github.io/GEAR-SONIC/). ## Evaluation Results | Task Family | Policy Setup | Task Specification | Logged Trials | Successes | Success Rate | | --- | --- | --- | ---: | ---: | ---: | | Walk to table and pick up object | Single mixed-object pickup policy | Walk to the table, localize a target object, grasp it, and lift it. Evaluated on 8 objects across 3 table heights and 9 object placements. Autonomous retries count as success. | 216 | 159 | 73.6% | | Soda can from small table to trash can | Task-specific soda-can policy | Walk toward a small table, pick up a soda can, rotate toward the trash can, step on the trigger, and drop the can inside. | 12 | 8 | 66.7% | For object pickup, each object was evaluated with 27 trials: 3 table heights (24, 27, and 30 inches) crossed with a 3 by 3 grid of object placements on the table (left/middle/right and front/middle/back). The failures were mostly grasp failures: the gripper missed the object, contacted it from an unstable grasp point, pushed it out of reach, or dropped it into a pose that the policy could not recover from. The soda-can failures were similarly dominated by missed grasps, unstable grasps, or missed trigger steps. ### Object Pickup Breakdown | Object | Trials | Successes | Success Rate | | --- | ---: | ---: | ---: | | Towel | 27 | 24 | 88.9% | | Shoe | 27 | 14 | 51.9% | | Apple | 27 | 19 | 70.4% | | Scoop | 27 | 18 | 66.7% | | Lamp | 27 | 17 | 63.0% | | Flashlight | 27 | 20 | 74.1% | | Fruit | 27 | 26 | 96.3% | | Sock | 27 | 21 | 77.8% | ## Demo Videos The MP4 samples below are attached as video files in this repository. Click any preview image to open the corresponding MP4. The source tree intentionally keeps only a curated sample set; full per-trial review videos are better shared as supplementary material through a stable gallery or archive. For more task demos beyond these two quantified results, see the [GEAR-SONIC project page](https://nvlabs.github.io/GEAR-SONIC/). ### Walk To Table And Pick Up Objects Representative examples are included for the evaluated object pickup task. Captions describe the behavior visible in the clip; some successful clips include retries, regrasping, object contact, or small recovery motions because autonomous recovery was counted as success during evaluation. A trial is considered failure when it stucks for over 30s without progress. | Object | Example 1 | Example 2 | | --- | --- | --- | | Lamp | [Lamp pickup from a 24-inch table placement](media/g1_real_eval/videos/mixed_pickup_lamp_01_pickup_success.mp4)
Picks up the lamp from a 24-inch table | [Lamp base grasp after contact at a 30-inch table placement](media/g1_real_eval/videos/mixed_pickup_lamp_02_base_grasp_success.mp4)
Contacts the lamp, then grasps the base | | Towel | [Towel pickup from a 24-inch table placement](media/g1_real_eval/videos/mixed_pickup_towel_01_pickup_success.mp4)
Picks up the towel | [Towel grasp, drop, and regrasp sequence](media/g1_real_eval/videos/mixed_pickup_towel_02_regrasp_success.mp4)
Grasps, drops, and regrasps | | Apple | [Apple pickup after a retry grasp](media/g1_real_eval/videos/mixed_pickup_apple_01_retry_success.mp4)
Retries the grasp and succeeds | [Apple pickup after gripper adjustment over the object](media/g1_real_eval/videos/mixed_pickup_apple_02_adjust_success.mp4)
Adjusts over the apple before lifting | | Shoe | [Shoe pickup after multiple grasp attempts](media/g1_real_eval/videos/mixed_pickup_shoe_01_many_attempts_success.mp4)
Succeeds after several attempts | [Shoe drop and regrasp success](media/g1_real_eval/videos/mixed_pickup_shoe_02_regrasp_success.mp4)
Drops the shoe, then regrasps | | Scoop | [Scoop pickup from a 27-inch table placement](media/g1_real_eval/videos/mixed_pickup_scoop_01_pickup_success.mp4)
Picks up the scoop | [Scoop pickup with noisy approach motion](media/g1_real_eval/videos/mixed_pickup_scoop_02_noisy_success.mp4)
Noisy approach, successful pickup | | Flashlight | [Smooth flashlight pickup](media/g1_real_eval/videos/mixed_pickup_flashlight_01_smooth_success.mp4)
Smooth pickup | [Flashlight pickup from a 27-inch table placement](media/g1_real_eval/videos/mixed_pickup_flashlight_02_pickup_success.mp4)
Picks up the flashlight | | Fruit | [Fruit pickup from a 24-inch table placement](media/g1_real_eval/videos/mixed_pickup_fruit_01_pickup_success.mp4)
Picks up the fruit | [Fruit grasp, drop, and regrasp success](media/g1_real_eval/videos/mixed_pickup_fruit_02_regrasp_success.mp4)
Grasps, drops, and regrasps | | Sock | [Sock top-down grasp success](media/g1_real_eval/videos/mixed_pickup_sock_01_top_grasp_success.mp4)
Top-down grasp succeeds | [Sock grasp, drop, and regrasp success](media/g1_real_eval/videos/mixed_pickup_sock_02_regrasp_success.mp4)
Drops once, then regrasps | ### Soda Can From Small Table To Trash Can | Task | Example 1 | Example 2 | Example 3 | | --- | --- | --- | --- | | Soda can to trash | [Soda can picked up and dropped into the trash can in one smooth sequence](media/g1_real_eval/videos/soda_can_table_trash_01_smooth_success.mp4)
Smooth full sequence | [Soda can succeeds after a second grasp and second trigger step](media/g1_real_eval/videos/soda_can_table_trash_02_second_grasp_step_success.mp4)
Second grasp and trigger step | [Soda can picked up and dropped into the trash can without a visible retry](media/g1_real_eval/videos/soda_can_table_trash_03_clean_success.mp4)
Single-attempt full sequence | ## Data Collection Experience We followed the [GR00T Whole-Body Control data collection tutorial](https://nvlabs.github.io/GR00T-WholeBodyControl/tutorials/data_collection.html) to collect G1 whole-body manipulation datasets. Training a good policy depends on both collecting high-quality data and configuring training properly. Each demonstration should ideally complete the task on the first attempt, without corrective motions such as re-grasping after a failed grasp or redoing a missed stepping trigger. For the mixed object pickup policy, we trained for 60k iterations at batch size 256 on roughly 18k episodes covering about 50 different objects; for the soda-can-to-trash policy, we trained for 20k iterations at batch size 32 on roughly 150 episodes of this single task. ### Notes 1. The mixed pickup result uses one policy trained from a combined object dataset and evaluated across multiple table-top objects. 2. For the pickup task, we used an in-house UMI gripper design. Similar tasks may require users to build task-appropriate gripper designs for their own hardware setup. ## Data-Train-Eval Workflow This benchmark used the public GR00T N1.7 and GR00T Whole-Body Control workflow: 1. Collect G1 demonstrations with SONIC teleoperation. 2. Fine-tune GR00T N1.7 with `UNITREE_G1_SONIC`. 3. Run the GR00T policy server and the SONIC robot-side controller. 4. Evaluate in closed loop on the real robot with video recording. Fine-tuning used the SONIC embodiment tag: ```bash bash examples/finetune.sh \ --base-model-path nvidia/GR00T-N1.7-3B \ --dataset-path /path/to/your/lerobot_dataset \ --embodiment-tag UNITREE_G1_SONIC \ --output-dir /path/to/output_checkpoint \ --experiment-name g1-sonic-task ``` Closed-loop evaluation used a GR00T policy server: ```bash python gr00t/eval/run_gr00t_server.py \ --model-path /path/to/output_checkpoint/checkpoint- \ --embodiment-tag UNITREE_G1_SONIC \ --device cuda:0 \ --host 0.0.0.0 \ --port 5550 ``` The robot-side controller follows the GR00T Whole-Body Control VLA inference workflow. ### Notes - The data format should be LeRobot V2 already and ready to train the model. - It would be beneficial to check open-loop robot joint trajectory matching against training set/validation set before deploying on real robot. - As suggested in Data Collection Experience, data quality is preferred than quantity during post-training. ## References - [NVIDIA Isaac GR00T](https://github.com/NVIDIA/Isaac-GR00T) - [SONIC paper](https://nvlabs.github.io/GEAR-SONIC/static/pdf/sonic_paper.pdf) - [GR00T Whole-Body Control documentation](https://nvlabs.github.io/GR00T-WholeBodyControl/) - [GEAR-SONIC project page](https://nvlabs.github.io/GEAR-SONIC/) - [GR00T data preparation guide](../../getting_started/data_preparation.md) - [GR00T policy server/client guide](../../getting_started/policy.md)