sheepsky commited on
Commit
ffcd916
·
verified ·
1 Parent(s): ede2095

Update X-NavDP README

Browse files
Files changed (1) hide show
  1. README.md +241 -3
README.md CHANGED
@@ -1,3 +1,241 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <p align="center">
2
+ <h1 align="center"><strong>X-NavDP</strong></h1>
3
+ <h2 align="center">Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching</h2>
4
+ <p align="center">
5
+ <a href="https://yty-sky.github.io/" target="_blank">Tianyu Yang</a><sup>1,2*</sup>&emsp;
6
+ <a href="https://jzengym.github.io/JZENGYM/" target="_blank">Yiming Zeng</a><sup>3,2*</sup>&emsp;
7
+ <a href="https://wzcai99.github.io/" target="_blank">Wenzhe Cai</a><sup>2*</sup>&emsp;
8
+ <a href="https://yuqiang-yang.github.io/" target="_blank">Yuqiang Yang</a><sup>2</sup>&emsp;
9
+ <a href="https://steinate.github.io/" target="_blank">Jiaqi Peng</a><sup>4,2</sup>&emsp;
10
+ <a href="https://scholar.google.com/citations?view_op=search_authors&mauthors=Hui+Cheng+Sun+Yat-sen+University&hl=en" target="_blank">Hui Cheng</a><sup>3</sup>&emsp;
11
+ <a href="https://oceanpang.github.io/" target="_blank">Jiangmiao Pang</a><sup>2</sup>&emsp;
12
+ <a href="https://tai-wang.github.io/" target="_blank">Tai Wang</a><sup>2&dagger;</sup>
13
+ </p>
14
+ <p align="center">
15
+ <sup>1</sup>Fudan University&emsp;
16
+ <sup>2</sup>Shanghai AI Laboratory&emsp;
17
+ <sup>3</sup>Sun Yat-sen University&emsp;
18
+ <sup>4</sup>Tsinghua University
19
+ </p>
20
+ </p>
21
+
22
+ <div id="top" align="center">
23
+
24
+ [![Project](https://img.shields.io/badge/Project-9c403d?style=flat)](https://yty-sky.github.io/x-navdp-project-page/)
25
+ [![arXiv](https://img.shields.io/badge/arXiv-coming%20soon-3b6291?style=flat)](https://yty-sky.github.io/x-navdp-project-page/#bibtex)
26
+ [![Video](https://img.shields.io/badge/Video-c97937?style=flat)](https://yty-sky.github.io/x-navdp-project-page/#video)
27
+ [![Benchmark](https://img.shields.io/badge/Benchmark-8A2BE2?style=flat)](../../README.md#-internvla-n1-system-1-benchmark)
28
+ [![Dataset](https://img.shields.io/badge/Dataset-548B54?style=flat)](https://huggingface.co/datasets/InternRobotics/Scene-N1/tree/main/n1_eval_scenes)
29
+ [![GitHub star chart](https://img.shields.io/github/stars/InternRobotics/NavDP?style=square)](https://github.com/InternRobotics/NavDP)
30
+
31
+ </div>
32
+
33
+ <p align="center">
34
+ <a href="#installation">Installation</a> |
35
+ <a href="#assets">Assets</a> |
36
+ <a href="#training">Training</a> |
37
+ <a href="#evaluation">Evaluation</a> |
38
+ <a href="#results">Results</a> |
39
+ <a href="#citation">Citation</a>
40
+ </p>
41
+
42
+ ## Introduction
43
+
44
+ X-NavDP post-trains a pretrained RGBD camera-based navigation diffusion policy via online reinforcement learning across heterogeneous embodiments. Beyond improving general navigation and obstacle avoidance performance, it gains new capabilities for backing out of traps, long-obstacle detours, and embodiment-aware behavior adaptation. Our contributions to navigation diffusion policy are threefold:
45
+
46
+ - &#9889; **Data-efficient RL post-training.** X-NavDP enhances pretrained diffusion policies through efficient large-scale, multi-scene online post-training for stronger general navigation ability.
47
+ - &#9874; **Structured exploration and stable training.** Goal-agnostic diffusion trajectories and **Group Q-score Reweighted Matching** enable structured exploration, improve training stability, and handle hard cases.
48
+ - &#129309; **Cross-robot generalization and temporal consistency.** Lightweight embodiment modulation and RTC guidance improve cross-robot generalization and temporal consistency, leading to superior post-trained navigation performance.
49
+
50
+ <p align="center">
51
+ <img src="fig/teaser.jpg" width="80%">
52
+ </p>
53
+
54
+ ## Installation
55
+
56
+ Clone NavDP and enter the self-contained X-NavDP baseline:
57
+
58
+ ```bash
59
+ git clone https://github.com/InternRobotics/NavDP.git
60
+ cd NavDP/baselines/x-navdp
61
+ ```
62
+
63
+ Create a Python environment and install the dependencies:
64
+
65
+ ```bash
66
+ conda create -n navrl python=3.11
67
+ conda activate navrl
68
+ pip install -r requirements.txt
69
+ ```
70
+
71
+ Install [Isaac Sim 5.0.0](https://docs.isaacsim.omniverse.nvidia.com/5.0.0/installation/index.html), [Isaac Lab 0.46.2](https://isaac-sim.github.io/IsaacLab/main/source/setup/installation/index.html), and `isaaclab-rl==0.4.0` following their official instructions. Then install acados following the [acados installation guide](https://docs.acados.org/installation/index.html) and [Python interface guide](https://docs.acados.org/python_interface/index.html). The acados setup is lightweight and mainly requires a CMake build plus the Python interface. Before running training or evaluation, expose the acados libraries and accept the Omniverse EULA:
72
+
73
+ ```bash
74
+ export ACADOS_SOURCE_DIR=/path/to/acados
75
+ export LD_LIBRARY_PATH="${ACADOS_SOURCE_DIR}/lib:${LD_LIBRARY_PATH:-}"
76
+ export OMNI_KIT_ACCEPT_EULA=YES
77
+ export OMNI_KIT_ALLOW_ROOT=1
78
+ ```
79
+
80
+ ## Assets
81
+
82
+ Large assets, datasets, robot USDs, low-level controller checkpoints, and pretrained policy checkpoints are not included in this repository. You need to download these assets manually before running training or evaluation.
83
+
84
+ The **GRScenes100 / N1 evaluation scenes** are available from [InternRobotics/Scene-N1](https://huggingface.co/datasets/InternRobotics/Scene-N1/tree/main/n1_eval_scenes). This dataset contains sky textures, materials, cluttered easy/hard scenes, InternScenes home scenes, and InternScenes commercial scenes.
85
+
86
+ The **X-NavDP assets** are available from [InternRobotics/X-NavDP](https://huggingface.co/InternRobotics/X-NavDP/tree/main). This repository provides `navigation_metadata`, robot assets, low-level controller checkpoints, `scene_split.json`, the NavDP pretrained checkpoint, and the X-NavDP post-trained checkpoint.
87
+
88
+ After downloading, place or symlink the scene data, metadata, robot assets, and checkpoints in the following layout. The scene root can also be kept elsewhere and passed with `SCENE_DIR`.
89
+
90
+ <table>
91
+ <tr>
92
+ <td width="52%" valign="top">
93
+
94
+ **Scene data and navigation metadata**
95
+
96
+ ```text
97
+ x-navdp/
98
+ +-- data/scenes/
99
+ +-- scene_split.json
100
+ +-- SkyTexture/
101
+ +-- Materials/
102
+ +-- cluttered_easy/
103
+ +-- cluttered_hard/
104
+ +-- internscenes_commercial/
105
+ | +-- models/
106
+ | +-- Materials/
107
+ | +-- scenes_commercial/
108
+ +-- internscenes_home/
109
+ | +-- models/
110
+ | +-- Materials/
111
+ | +-- scenes_home/
112
+ +-- navigation_metadata/
113
+ +-- internscenes_commercial/
114
+ +-- internscenes_home/
115
+ +-- cluttered_easy/
116
+ +-- cluttered_hard/
117
+ ```
118
+
119
+ </td>
120
+ <td width="48%" valign="top">
121
+
122
+ **Code, robot assets, and checkpoints**
123
+
124
+ ```text
125
+ x-navdp/
126
+ +-- train.py
127
+ +-- config/
128
+ +-- scripts/
129
+ +-- src/
130
+ | +-- environment/
131
+ | | +-- controllers/checkpoints/
132
+ | | +-- humanoid_g1/policy.pt
133
+ | | +-- quadruped_go2/policy.pt
134
+ | +-- x_navdp/
135
+ | +-- training/
136
+ | +-- utils/
137
+ +-- eval/
138
+ +-- data/robots/
139
+ | +-- dingo.usd
140
+ | +-- unitreeg1.usd
141
+ +-- pretrain_model/
142
+ | +-- navdp_pretrained.ckpt
143
+ +-- checkpoints/
144
+ +-- x-navdp_posttrain.ckpt
145
+ ```
146
+
147
+ </td>
148
+ </tr>
149
+ </table>
150
+
151
+ Training and evaluation expect `SCENE_DIR` to point to the scene root. The training config initializes from `pretrain_model/navdp_pretrained.ckpt`; evaluation examples use the released post-trained checkpoint under `checkpoints/`. Dingo and G1 load robot USDs from `data/robots/`, while Unitree Go2 uses the Isaac Lab asset path by default.
152
+
153
+ ## Training
154
+
155
+ The released [post-trained model](https://huggingface.co/InternRobotics/X-NavDP/blob/main/x-navdp_posttrain.ckpt) was trained for 24,000 steps on 72 scenes, including 16 clutter scenes and 56 home/commercial scenes.
156
+
157
+ Single-node 8-GPU distributed training can be launched with:
158
+
159
+ ```bash
160
+ export SCENE_DIR=/path/to/NavDP/baselines/x-navdp/data/scenes
161
+ export NPROC_PER_NODE=8
162
+
163
+ bash scripts/run_ddp_train.sh \
164
+ --config_file config/x-navdp_config.yaml
165
+ ```
166
+
167
+ For a smaller debug run, reduce the number of processes and environments:
168
+
169
+ ```bash
170
+ export NPROC_PER_NODE=1
171
+
172
+ bash scripts/run_ddp_train.sh \
173
+ --num_envs 1 \
174
+ --embodiments dingo \
175
+ --max_steps 100
176
+ ```
177
+
178
+ Training writes periodic evaluation records to the directory passed through
179
+ `--txt_dir`. Aggregate these logs into global, per-embodiment, and per-scene
180
+ success-rate CSV files and plots with:
181
+
182
+ ```bash
183
+ python scripts/aggregate_success.py ./txt/x-navdp
184
+ ```
185
+
186
+ By default, outputs are written to `./result/<txt_subdir_name>/`; for the
187
+ command above, the output directory is `./result/x-navdp/`.
188
+ Each log row has the form
189
+ `rank,step,episode,success,trainer_success_rate`. Home/commercial and clutter
190
+ results are reported separately. Use `--output-dir` to override the result
191
+ directory, or `--no-plots` to generate CSV files only; run
192
+ `python scripts/aggregate_success.py --help` for EMA and milestone options.
193
+
194
+ ## Evaluation
195
+
196
+ Evaluation uses a policy server and an Isaac Lab client.
197
+
198
+ Start the policy server:
199
+
200
+ ```bash
201
+ bash eval/scripts/start_policy_server.sh \
202
+ --checkpoint checkpoints/x-navdp_posttrain.ckpt \
203
+ --embodiment quadruped
204
+ ```
205
+
206
+ Run point-goal evaluation:
207
+
208
+ ```bash
209
+ bash eval/scripts/run_evaluation.sh \
210
+ --config_file eval/config/eval_pointgoal/quadruped_internscene_commercial.yaml
211
+ ```
212
+
213
+ By default, evaluation runs all start-goal samples in the selected scene. Use `--num_episodes` to evaluate a subset and `--max_steps` to cap the total simulation steps for smoke tests or debugging.
214
+
215
+ Evaluation configs are provided under `eval/config/eval_pointgoal/`, and outputs are written under `outputs/evaluation/<embodiment>_<scene_type>/`. Each scene directory contains a `metric.csv` file whose first two columns are success and SPL. To print per-USD scene SR and SPL, run:
216
+
217
+ ```bash
218
+ python eval/scripts/stat_eval_metrics.py outputs/evaluation/quadruped_commercial
219
+ ```
220
+
221
+ ## Results
222
+
223
+ On the project benchmark, X-NavDP improves the overall simulation success rate from 61.20% to 84.28% and improves real-world hard-case success rate from 10% to 65%. See the [project results](https://yty-sky.github.io/x-navdp-project-page/#results) for simulation and real-world comparisons across wheeled Dingo, quadruped Unitree Go2, and humanoid Unitree G1 embodiments.
224
+
225
+ ## Citation
226
+
227
+ ```bibtex
228
+ @misc{yang2026xnavdp,
229
+ title = {X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching},
230
+ author = {Tianyu Yang and Yiming Zeng and Wenzhe Cai and Yuqiang Yang and Jiaqi Peng and Hui Cheng and Jiangmiao Pang and Tai Wang},
231
+ year = {2026}
232
+ }
233
+ ```
234
+
235
+ ## License
236
+
237
+ Original X-NavDP code is released under the [MIT License](LICENSE). Vendored and external dependencies retain their upstream terms; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
238
+
239
+ ## Acknowledgement
240
+
241
+ This project builds on Isaac Sim, Isaac Lab, acados, and NavDP. X-NavDP is distributed as a self-contained baseline in the [InternRobotics/NavDP](https://github.com/InternRobotics/NavDP) repository.