Commit ·
1e68f76
1
Parent(s): 689cce6
Link to paper, project page, and add citation (#1)
Browse files- Link to paper, project page, and add citation (f0f1a345cd76a8c362128b0631c71669f5a0b548)
Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
library_name: diffsynth
|
|
|
|
| 4 |
pipeline_tag: robotics
|
| 5 |
tags:
|
| 6 |
- robotics
|
|
@@ -13,9 +13,10 @@ tags:
|
|
| 13 |
|
| 14 |
# FlowWAM
|
| 15 |
|
| 16 |
-
Checkpoints for **FlowWAM** — a dual-stream (RGB + optical-flow) Wan2.2 world
|
| 17 |
-
model paired with an IDM action expert for RoboTwin dual-arm manipulation.
|
| 18 |
|
|
|
|
|
|
|
| 19 |
- **Code:** https://github.com/YixiangChen515/FlowWAM
|
| 20 |
- **Dataset:** https://huggingface.co/datasets/YixiangChen/FlowWAM_RoboTwin
|
| 21 |
|
|
@@ -29,16 +30,11 @@ model paired with an IDM action expert for RoboTwin dual-arm manipulation.
|
|
| 29 |
|
| 30 |
## RoboTwin flow-action IDM checkpoint
|
| 31 |
|
| 32 |
-
`flowwam_robotwin.safetensors` jointly denoises an RGB stream and a head-camera
|
| 33 |
-
optical-flow stream, and predicts a 14-D dual-arm action chunk by cross-attending
|
| 34 |
-
to the video DiT's per-layer hidden states. It is trained on the RoboTwin 2.0
|
| 35 |
-
`aloha-agilex` demonstrations (see the dataset repo above).
|
| 36 |
|
| 37 |
### Inference
|
| 38 |
|
| 39 |
-
Keep `flowwam_robotwin.safetensors` and `flowwam_robotwin_action_norm_stats.npz`
|
| 40 |
-
**in the same folder** — the server derives the norm-stats path from the
|
| 41 |
-
checkpoint path. The decoding config must match the training config:
|
| 42 |
|
| 43 |
| Setting | Value |
|
| 44 |
|---|---|
|
|
@@ -59,6 +55,20 @@ CHECKPOINT=/path/to/flowwam_robotwin.safetensors bash start_server.sh
|
|
| 59 |
|
| 60 |
See the code repo's `inference/README.md` for the full RoboTwin evaluation setup.
|
| 61 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
## License
|
| 63 |
|
| 64 |
-
Released under the Apache-2.0 License.
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
library_name: diffsynth
|
| 3 |
+
license: apache-2.0
|
| 4 |
pipeline_tag: robotics
|
| 5 |
tags:
|
| 6 |
- robotics
|
|
|
|
| 13 |
|
| 14 |
# FlowWAM
|
| 15 |
|
| 16 |
+
Checkpoints for **FlowWAM** — a dual-stream (RGB + optical-flow) Wan2.2 world model paired with an IDM action expert for RoboTwin dual-arm manipulation.
|
|
|
|
| 17 |
|
| 18 |
+
- **Paper:** [FlowWAM: Optical Flow as a Unified Action Representation for World Action Models](https://huggingface.co/papers/2607.13017)
|
| 19 |
+
- **Project Page:** https://flow-wam.github.io/
|
| 20 |
- **Code:** https://github.com/YixiangChen515/FlowWAM
|
| 21 |
- **Dataset:** https://huggingface.co/datasets/YixiangChen/FlowWAM_RoboTwin
|
| 22 |
|
|
|
|
| 30 |
|
| 31 |
## RoboTwin flow-action IDM checkpoint
|
| 32 |
|
| 33 |
+
`flowwam_robotwin.safetensors` jointly denoises an RGB stream and a head-camera optical-flow stream, and predicts a 14-D dual-arm action chunk by cross-attending to the video DiT's per-layer hidden states. It is trained on the RoboTwin 2.0 `aloha-agilex` demonstrations (see the dataset repo above).
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
### Inference
|
| 36 |
|
| 37 |
+
Keep `flowwam_robotwin.safetensors` and `flowwam_robotwin_action_norm_stats.npz` **in the same folder** — the server derives the norm-stats path from the checkpoint path. The decoding config must match the training config:
|
|
|
|
|
|
|
| 38 |
|
| 39 |
| Setting | Value |
|
| 40 |
|---|---|
|
|
|
|
| 55 |
|
| 56 |
See the code repo's `inference/README.md` for the full RoboTwin evaluation setup.
|
| 57 |
|
| 58 |
+
## Citation
|
| 59 |
+
|
| 60 |
+
```bibtex
|
| 61 |
+
@misc{flowwam,
|
| 62 |
+
title={FlowWAM: Optical Flow as a Unified Action Representation for World Action Models},
|
| 63 |
+
author={Yixiang Chen and Peiyan Li and Yuan Xu and Qisen Ma and Jiabing Yang and Kai Wang and Jianhua Yang and Dong An and He Guan and Gaoteng Liu and Jianlou Si and Jun Huang and Jing Liu and Nianfeng Liu and Yan Huang and Liang Wang},
|
| 64 |
+
year={2026},
|
| 65 |
+
eprint={2607.13017},
|
| 66 |
+
archivePrefix={arXiv},
|
| 67 |
+
primaryClass={cs.RO},
|
| 68 |
+
url={https://arxiv.org/abs/2607.13017},
|
| 69 |
+
}
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
## License
|
| 73 |
|
| 74 |
+
Released under the Apache-2.0 License.
|