BreezeWu commited on
Commit
e68be84
Β·
verified Β·
1 Parent(s): 9e16b58

Update model card: multi-deployment (torch/onnx) support

Browse files
Files changed (1) hide show
  1. README.md +34 -43
README.md CHANGED
@@ -8,61 +8,52 @@ tags:
8
  - robotics
9
  - IB-Robot
10
  - ascend
 
11
  - edge-deployment
12
  ---
13
 
14
  # Model Card for FullSubNet (IB-Robot)
15
 
16
- FullSubNet speech enhancement model with cumulative Laplace normalization, compiled as stateful Ascend 310P OM artifacts for streaming 4-channel microphone array processing. Packaged for the [IB-Robot](https://gitcode.com/openeuler/IB_Robot) framework.
 
 
 
17
 
18
- ## Repository Structure
19
-
20
- - `inference_manifest.json` β€” deployment routing (schema v3, stateful stream contract)
21
- - `assets/adapter.json` β€” algorithm contract (n_fft=512, hop=256, cumulative_laplace_norm)
22
- - `artifacts/ascend/fullsubnet/` β€” two stateful OM modules:
23
- - `fullsubnet_cum_stateful_fb_b4_t2_fp16.om` β€” full-band encoder (batch=4, time_steps=2)
24
- - `fullsubnet_cum_stateful_sb_b4_t2_fp16.om` β€” sub-band decoder (batch=1028, time_steps=2)
25
-
26
- ## Deployment
27
 
28
- | Target | Backend | Runtime | Hardware |
29
- |--------|---------|---------|----------|
30
- | `ascend_310p` | ascend | ACL (cann-8.1.RC1) | Ascend 310P1 |
 
 
31
 
32
- **Execution:** stateful 2-stage pipeline `fullsubnet_fb` -> `fullsubnet_sb` with LSTM hidden/cell state banks (double-buffered, runtime-exclusive).
 
 
 
33
 
34
- **Pipeline:**
35
- 1. 4-channel audio (512 samples) -> STFT (512-pt, hop 256) -> magnitude [4,2,257]
36
- 2. Cumulative Laplace normalization
37
- 3. FB OM -> features [4,2,257] + LSTM state
38
- 4. Host-side sub-band feature construction -> [1028,2,32]
39
- 5. SB OM -> compressed complex mask [1028,2,2]
40
- 6. CIRM decompression -> iSTFT/OLA -> enhanced audio [512,4]
41
 
42
- **Input:** `observation.audio_4ch` float32 [-1,4]
43
- **Output:** `voice.audio_enhanced_4ch` float32 [-1,4]
 
 
 
 
44
 
45
- ## Source Model
 
 
46
 
47
- This bundle's OM artifacts were compiled from the upstream FullSubNet checkpoint:
48
-
49
- - **Source checkpoint:** [cum_fullsubnet_best_model_218epochs.tar](https://github.com/Audio-WestlakeU/FullSubNet/releases/download/v0.2/cum_fullsubnet_best_model_218epochs.tar) (218 epochs, 5.6M params)
50
- - **Source repository:** [github.com/Audio-WestlakeU/FullSubNet](https://github.com/Audio-WestlakeU/FullSubNet)
51
-
52
- Download script: `scripts/download_speech_direction_models.sh`. The OM artifacts were compiled from the torch checkpoint via ATC for Ascend 310P1 (cann-8.1.RC1, fp16).
53
 
54
- > **Note:** Must use `cumulative_laplace_norm` normalization. Not compatible with `offline_laplace_norm` configurations.
 
 
 
55
 
56
- ## Citation
57
 
58
- @inproceedings{fullsubnet,
59
- title = {FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Speech Enhancement},
60
- author = {Chen, Jun and Du, Zilin and others},
61
- booktitle = {ICASSP},
62
- year = {2022}
63
- }
64
- @software{ib_robot,
65
- title = {IB-Robot: Intelligence Boom Robot},
66
- url = {https://gitcode.com/openeuler/IB_Robot},
67
- license = {Apache-2.0}
68
- }
 
8
  - robotics
9
  - IB-Robot
10
  - ascend
11
+ - torch
12
  - edge-deployment
13
  ---
14
 
15
  # Model Card for FullSubNet (IB-Robot)
16
 
17
+ FullSubNet speech enhancement model with cumulative Laplace normalization for
18
+ streaming 4-channel microphone array processing, packaged for the
19
+ [IB-Robot](https://atomgit.com/openeuler/IB_Robot) framework with three
20
+ deployments sharing one stream contract.
21
 
22
+ ## Deployments
 
 
 
 
 
 
 
 
23
 
24
+ | deployment | backend | artifacts | notes |
25
+ |---|---|---|---|
26
+ | `ascend_310p` | Ascend ACL (Ascend310P1) | stateful FB/SB OM pair | board-side streaming (LSTM state via state_links) |
27
+ | `torch_cuda` | PyTorch CUDA | β€” (ckpt in `assets/`) | host-side streaming executor |
28
+ | `torch_cpu` | PyTorch CPU | β€” (ckpt in `assets/`) | CPU fallback (watch the 128 ms hop budget) |
29
 
30
+ All deployments share `tensor_model/fullsubnet/enhance`
31
+ (`observation.audio_4ch [-1,4] -> voice.audio_enhanced_4ch [-1,4]`) with a
32
+ stateful stream execution contract (`state_bank_mode: runtime_exclusive`,
33
+ `max_open_streams: 1`).
34
 
35
+ ## Weights provenance
 
 
 
 
 
 
36
 
37
+ - `assets/cum_fullsubnet_best_model_218epochs.tar` = Audio-WestlakeU/FullSubNet
38
+ official v0.2 release checkpoint (sha256
39
+ `d08d09107eb276b8dc3d2d9fff995f4354a51fa3347125f52f8b9aea7c339f81`)
40
+ - The 310P stateful OM pair was converted from the same checkpoint
41
+ - `assets/cum_fullsubnet_best_model_218epochs.manifest.json` pins the digest and
42
+ `norm_type: cumulative_laplace_norm` (do not mix with offline-norm checkpoints)
43
 
44
+ Note: the `.tar` file is the upstream PyTorch serialization container
45
+ (legacy `torch.save` format), loaded directly by `torch.load` β€” no extraction
46
+ step.
47
 
48
+ ## Repository Structure
 
 
 
 
 
49
 
50
+ - `inference_manifest.json` β€” deployment routing (schema v3, stateful stream contract)
51
+ - `assets/adapter.json` β€” algorithm contract (STFT 512/256, T=2, look-ahead 2)
52
+ - `assets/cum_fullsubnet_best_model_218epochs.tar` β€” Torch checkpoint
53
+ - `artifacts/ascend/fullsubnet/*.om` β€” stateful FB/SB OM modules
54
 
55
+ ## Usage
56
 
57
+ Host: `voice_asr_service.speech_direction` with backend `stateful_torch_cuda` /
58
+ `stateful_torch_cpu`; board: select `ascend_310p` via the unified inference
59
+ runtime.