ElysiaTrue commited on
Commit
8dcc67c
Β·
verified Β·
1 Parent(s): d2c4b7e

Rename model card to Streaming-WAM

Browse files
Files changed (1) hide show
  1. README.md +23 -24
README.md CHANGED
@@ -11,25 +11,25 @@ tags:
11
  ---
12
 
13
  <div align="center">
14
- <h1>StreamWAM</h1>
15
  <h3>Streaming World-Action Models for Robotic Manipulation</h3>
16
 
17
- <a href="https://github.com/SJTU-DENG-Lab/StreamWAM"><img src="https://img.shields.io/badge/GitHub-Code-111827?logo=github" alt="GitHub Code"></a>
18
- <a href="https://github.com/SJTU-DENG-Lab/StreamWAM/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-6B5BFF" alt="Apache 2.0 License"></a>
19
  </div>
20
 
21
- StreamWAM is a research framework for streaming World-Action Models. It provides a unified testbed for studying and comparing efficient robot-control strategies.
22
 
23
- StreamWAM uses the actions currently being executed by the robot to guide its next prediction. This allows action execution and model inference to proceed together, reducing the time required to complete a robot task while maintaining strong control performance.
24
 
25
- This repository provides the released LIBERO checkpoints. The corresponding inference and evaluation code is available in the [StreamWAM GitHub repository](https://github.com/SJTU-DENG-Lab/StreamWAM).
26
 
27
  ## Released checkpoints
28
 
29
  | Directory | Model | Description |
30
  | --- | --- | --- |
31
  | `joint-cd/` | FastWAM-Joint-CD | Fast one-step joint world-and-action prediction for LIBERO. |
32
- | `ac-stream/` | StreamWAM | Recommended StreamWAM checkpoint for efficient LIBERO evaluation. |
33
 
34
  Each directory contains:
35
 
@@ -48,15 +48,15 @@ We evaluate all methods on LIBERO-10, LIBERO-Spatial, LIBERO-Goal, and LIBERO-Ob
48
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
49
  | FastWAM | 96.20 | 96.20 | 94.20 | 96.20 | 95.70 | 493.0 | 16.31 / 8.25 |
50
  | FastWAM-Joint-CD | 97.20 | 99.60 | 98.60 | 100.00 | 98.85 | 114.2 | 6.89 / 3.74 |
51
- | **StreamWAM** | **96.60** | **98.80** | **97.40** | **100.00** | **98.20** | **41.0** | **5.36 / 3.15** |
52
 
53
  ## Installation
54
 
55
  Clone the code repository and install the environment:
56
 
57
  ```bash
58
- git clone https://github.com/SJTU-DENG-Lab/StreamWAM.git
59
- cd StreamWAM
60
 
61
  python -m pip install -U uv
62
  uv sync
@@ -79,14 +79,14 @@ uv run huggingface-cli download Wan-AI/Wan2.2-TI2V-5B \
79
  Download both released checkpoints:
80
 
81
  ```bash
82
- hf download SJTU-DENG-Lab/StreamWAM \
83
- --local-dir checkpoints/streamwam
84
  ```
85
 
86
  The resulting layout is:
87
 
88
  ```text
89
- checkpoints/streamwam/
90
  β”œβ”€β”€ joint-cd/
91
  β”‚ β”œβ”€β”€ model.pt
92
  β”‚ └── dataset_stats.json
@@ -95,7 +95,7 @@ checkpoints/streamwam/
95
  └── dataset_stats.json
96
  ```
97
 
98
- ## Run StreamWAM
99
 
100
  Evaluate all 40 LIBERO tasks once on four GPUs:
101
 
@@ -104,9 +104,9 @@ PYTHON_BIN=.venv/bin/python \
104
  GPU_IDS=0,1,2,3 \
105
  BACKBONE_PATH="$PWD/checkpoints/Wan2.2-TI2V-5B" \
106
  LIBERO_HOME_PATH="$PWD/third_party/LIBERO" \
107
- CHECKPOINT_PATH="$PWD/checkpoints/streamwam/ac-stream/model.pt" \
108
- STATS_PATH="$PWD/checkpoints/streamwam/ac-stream/dataset_stats.json" \
109
- bash examples/libero/scripts/launch_streamwam_libero_ac_stream_4gpu.sh \
110
  --ac-stream-accelerated
111
  ```
112
 
@@ -119,11 +119,11 @@ python examples/libero/multigpu_rollout.py \
119
  --gpus 0,1,2,3 \
120
  --suites libero_spatial,libero_object,libero_goal,libero_10 \
121
  --num-trials 1 \
122
- --config examples/libero/configs/recipes/streamwam_libero_joint_cd_wan22_5b.yaml \
123
  --checkpoint-format fastwam \
124
- --checkpoint checkpoints/streamwam/joint-cd/model.pt \
125
  --backbone-path checkpoints/Wan2.2-TI2V-5B \
126
- --stats-path checkpoints/streamwam/joint-cd/dataset_stats.json \
127
  --libero-home third_party/LIBERO \
128
  --num-steps-wait 30 \
129
  --replan-steps 16 \
@@ -134,13 +134,12 @@ python examples/libero/multigpu_rollout.py \
134
  --save-video
135
  ```
136
 
137
- For more evaluation options, see the [LIBERO guide](https://github.com/SJTU-DENG-Lab/StreamWAM/blob/main/examples/libero/LIBERO.md).
138
 
139
  ## License
140
 
141
- Released under the [Apache License 2.0](https://github.com/SJTU-DENG-Lab/StreamWAM/blob/main/LICENSE).
142
 
143
  ## Acknowledgements
144
 
145
- StreamWAM builds on ideas and open-source work from [FastWAM](https://github.com/yuantianyuan01/FastWAM), [StarWAM](https://github.com/shaohua-pan/StarWAM), [LIBERO](https://github.com/Lifelong-Robot-Learning/LIBERO), and [Wan2.2](https://github.com/Wan-Video/Wan2.2).
146
-
 
11
  ---
12
 
13
  <div align="center">
14
+ <h1>Streaming-WAM</h1>
15
  <h3>Streaming World-Action Models for Robotic Manipulation</h3>
16
 
17
+ <a href="https://github.com/SJTU-DENG-Lab/Streaming-WAM"><img src="https://img.shields.io/badge/GitHub-Code-111827?logo=github" alt="GitHub Code"></a>
18
+ <a href="https://github.com/SJTU-DENG-Lab/Streaming-WAM/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache--2.0-6B5BFF" alt="Apache 2.0 License"></a>
19
  </div>
20
 
21
+ Streaming-WAM is a research framework for streaming World-Action Models. It provides a unified testbed for studying and comparing efficient robot-control strategies.
22
 
23
+ Streaming-WAM uses the actions currently being executed by the robot to guide its next prediction. This allows action execution and model inference to proceed together, reducing the time required to complete a robot task while maintaining strong control performance.
24
 
25
+ This repository provides the released LIBERO checkpoints. The corresponding inference and evaluation code is available in the [Streaming-WAM GitHub repository](https://github.com/SJTU-DENG-Lab/Streaming-WAM).
26
 
27
  ## Released checkpoints
28
 
29
  | Directory | Model | Description |
30
  | --- | --- | --- |
31
  | `joint-cd/` | FastWAM-Joint-CD | Fast one-step joint world-and-action prediction for LIBERO. |
32
+ | `ac-stream/` | Streaming-WAM | Recommended Streaming-WAM checkpoint for efficient LIBERO evaluation. |
33
 
34
  Each directory contains:
35
 
 
48
  | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
49
  | FastWAM | 96.20 | 96.20 | 94.20 | 96.20 | 95.70 | 493.0 | 16.31 / 8.25 |
50
  | FastWAM-Joint-CD | 97.20 | 99.60 | 98.60 | 100.00 | 98.85 | 114.2 | 6.89 / 3.74 |
51
+ | **Streaming-WAM** | **96.60** | **98.80** | **97.40** | **100.00** | **98.20** | **41.0** | **5.36 / 3.15** |
52
 
53
  ## Installation
54
 
55
  Clone the code repository and install the environment:
56
 
57
  ```bash
58
+ git clone https://github.com/SJTU-DENG-Lab/Streaming-WAM.git
59
+ cd Streaming-WAM
60
 
61
  python -m pip install -U uv
62
  uv sync
 
79
  Download both released checkpoints:
80
 
81
  ```bash
82
+ hf download SJTU-DENG-Lab/Streaming-WAM \
83
+ --local-dir checkpoints/streamingwam
84
  ```
85
 
86
  The resulting layout is:
87
 
88
  ```text
89
+ checkpoints/streamingwam/
90
  β”œβ”€β”€ joint-cd/
91
  β”‚ β”œβ”€β”€ model.pt
92
  β”‚ └── dataset_stats.json
 
95
  └── dataset_stats.json
96
  ```
97
 
98
+ ## Run Streaming-WAM
99
 
100
  Evaluate all 40 LIBERO tasks once on four GPUs:
101
 
 
104
  GPU_IDS=0,1,2,3 \
105
  BACKBONE_PATH="$PWD/checkpoints/Wan2.2-TI2V-5B" \
106
  LIBERO_HOME_PATH="$PWD/third_party/LIBERO" \
107
+ CHECKPOINT_PATH="$PWD/checkpoints/streamingwam/ac-stream/model.pt" \
108
+ STATS_PATH="$PWD/checkpoints/streamingwam/ac-stream/dataset_stats.json" \
109
+ bash examples/libero/scripts/launch_streamingwam_libero_ac_stream_4gpu.sh \
110
  --ac-stream-accelerated
111
  ```
112
 
 
119
  --gpus 0,1,2,3 \
120
  --suites libero_spatial,libero_object,libero_goal,libero_10 \
121
  --num-trials 1 \
122
+ --config examples/libero/configs/recipes/streamingwam_libero_joint_cd_wan22_5b.yaml \
123
  --checkpoint-format fastwam \
124
+ --checkpoint checkpoints/streamingwam/joint-cd/model.pt \
125
  --backbone-path checkpoints/Wan2.2-TI2V-5B \
126
+ --stats-path checkpoints/streamingwam/joint-cd/dataset_stats.json \
127
  --libero-home third_party/LIBERO \
128
  --num-steps-wait 30 \
129
  --replan-steps 16 \
 
134
  --save-video
135
  ```
136
 
137
+ For more evaluation options, see the [LIBERO guide](https://github.com/SJTU-DENG-Lab/Streaming-WAM/blob/main/examples/libero/LIBERO.md).
138
 
139
  ## License
140
 
141
+ Released under the [Apache License 2.0](https://github.com/SJTU-DENG-Lab/Streaming-WAM/blob/main/LICENSE).
142
 
143
  ## Acknowledgements
144
 
145
+ Streaming-WAM builds on ideas and open-source work from [FastWAM](https://github.com/yuantianyuan01/FastWAM), [StarWAM](https://github.com/shaohua-pan/StarWAM), [LIBERO](https://github.com/Lifelong-Robot-Learning/LIBERO), and [Wan2.2](https://github.com/Wan-Video/Wan2.2).