File size: 4,324 Bytes
700dd75 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 | # Decoupled WBC
Software stack for loco-manipulation experiments across multiple humanoid platforms, with primary support for the Unitree G1. This repository provides whole-body control policies, a teleoperation stack, and a data exporter.
---
## System Installation
### Prerequisites
- Ubuntu 22.04
- NVIDIA GPU with a recent driver
- Docker and NVIDIA Container Toolkit (required for GPU access inside the container)
### Repository Setup
Install Git and Git LFS:
```bash
sudo apt update
sudo apt install git git-lfs
git lfs install
```
Clone the repository:
```bash
mkdir -p ~/Projects
cd ~/Projects
git clone https://github.com/NVlabs/GR00T-WholeBodyControl.git
cd decoupled_wbc
```
### Docker Environment
We provide a Docker image with all dependencies pre-installed.
Install a fresh image and start a container:
```bash
./docker/run_docker.sh --install --root
```
This pulls the latest `decoupled_wbc` image from `docker.io/nvgear`.
Start or re-enter a container:
```bash
./docker/run_docker.sh --root
```
Use `--root` to run as the `root` user. To run as a normal user, build the image locally:
```bash
./docker/run_docker.sh --build
```
---
## Running the Control Stack
Once inside the container, the control policies can be launched directly.
- Simulation:
```bash
python decoupled_wbc/control/main/teleop/run_g1_control_loop.py
```
- Real robot: Ensure the host machine network is configured per the [G1 SDK Development Guide](https://support.unitree.com/home/en/G1_developer) and set a static IP at `192.168.123.222`, subnet mask `255.255.255.0`:
```bash
python decoupled_wbc/control/main/teleop/run_g1_control_loop.py --interface real
```
Keyboard shortcuts (terminal window):
- `]`: Activate policy
- `o`: Deactivate policy
- `9`: Release / Hold the robot
- `w` / `s`: Move forward / backward
- `a` / `d`: Strafe left / right
- `q` / `e`: Rotate left / right
- `z`: Zero navigation commands
- `1` / `2`: Raise / lower the base height
- `backspace` (viewer): Reset the robot in the visualizer
---
## Running the Teleoperation Stack
The teleoperation policy primarily uses Pico controllers for coordinated hand and body control. It also supports other teleoperation devices, including LeapMotion and HTC Vive with Nintendo Switch Joy-Con controllers.
Keep `run_g1_control_loop.py` running, and in another terminal run:
```bash
python decoupled_wbc/control/main/teleop/run_teleop_policy_loop.py --hand_control_device=pico --body_control_device=pico
```
### Pico Setup and Controls
Configure the teleop app on your Pico headset by following the [XR Robotics guidelines](https://github.com/XR-Robotics).
The necessary PC software is pre-installed in the Docker container. Only the [XRoboToolkit-PC-Service](https://github.com/XR-Robotics/XRoboToolkit-PC-Service) component is needed.
Prerequisites: Connect the Pico to the same network as the host computer.
Controller bindings:
- `menu + left trigger`: Toggle lower-body policy
- `menu + right trigger`: Toggle upper-body policy
- `Left stick`: X/Y translation
- `Right stick`: Yaw rotation
- `L/R triggers`: Control hand grippers
Pico unit test:
```bash
python decoupled_wbc/control/teleop/streamers/pico_streamer.py
```
---
## Running the Data Collection Stack
Run the full stack (control loop, teleop policy, and camera forwarder) via the deployment helper:
```bash
python decoupled_wbc/scripts/deploy_g1.py \
--interface sim \
--camera_host localhost \
--sim_in_single_process \
--simulator robocasa \
--image-publish \
--enable-offscreen \
--env_name PnPBottle \
--hand_control_device=pico \
--body_control_device=pico
```
The `tmux` session `g1_deployment` is created with panes for:
- `control_data_teleop`: Main control loop, data collection, and teleoperation policy
- `camera`: Camera forwarder
- `camera_viewer`: Optional live camera feed
Operations in the `controller` window (`control_data_teleop` pane, left):
- `]`: Activate policy
- `o`: Deactivate policy
- `k`: Reset the simulation and policies
- `` ` ``: Terminate the tmux session
- `ctrl + d`: Exit the shell in the pane
Operations in the `data exporter` window (`control_data_teleop` pane, right top):
- Enter the task prompt
Operations on Pico controllers:
- `A`: Start/Stop recording
- `B`: Discard trajectory
|