claroche1 commited on
Commit
db59f15
·
verified ·
1 Parent(s): fb26c8e

add the Monarch block-count sweep: monarch_5 / monarch_10 / monarch_20

Browse files
.gitattributes CHANGED
@@ -46,3 +46,6 @@ blockdiag_8/g_best filter=lfs diff=lfs merge=lfs -text
46
  blockdiag_full/g_best filter=lfs diff=lfs merge=lfs -text
47
  blockdiag_fc/g_best filter=lfs diff=lfs merge=lfs -text
48
  wide_blockdiag/g_best filter=lfs diff=lfs merge=lfs -text
 
 
 
 
46
  blockdiag_full/g_best filter=lfs diff=lfs merge=lfs -text
47
  blockdiag_fc/g_best filter=lfs diff=lfs merge=lfs -text
48
  wide_blockdiag/g_best filter=lfs diff=lfs merge=lfs -text
49
+ monarch_10/g_best filter=lfs diff=lfs merge=lfs -text
50
+ monarch_20/g_best filter=lfs diff=lfs merge=lfs -text
51
+ monarch_5/g_best filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -70,6 +70,26 @@ does survive the fix — but it had never actually been tested before it.
70
  | `monarch_fc` | 2.38 M | 2.843 | 2.831 | +0.012 |
71
  | `monarch_full` | 1.10 M | 2.838 | 2.846 | −0.009 |
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ### Block-diagonal, dense, butterfly
74
 
75
  | run | params | FP32 PESQ | int8 PESQ | Δ (FP32→int8) |
@@ -94,7 +114,9 @@ does survive the fix — but it had never actually been tested before it.
94
  NSNet2 predicts a magnitude mask and reuses the noisy phase, which caps PESQ
95
  regardless of how expressive the mask predictor is. **The dense model was
96
  already over-parameterized** — which is exactly why aggressive structuring is
97
- nearly free. For deployment, take the smallest (`blockdiag_8` / `monarch_8`).
 
 
98
  - **Genuine Monarch beats block-diagonal, but marginally** (+0.011…+0.038 FP32 at
99
  matched `nblocks`) and it costs parameters — its second factor makes it larger.
100
  Consistent with the saturation above.
@@ -113,7 +135,8 @@ static int8 ONNX, and the exact `config.json` it was trained with.
113
  baseline/ blockdiag_8/ monarch_8/ butterfly_fc/
114
  blockdiag_fc/ blockdiag_full/ monarch_fc/ butterfly_full/
115
  wide_blockdiag/ monarch_full/ wide_monarch/ butterfly_ortho/
116
- butterfly_2blocks/
 
117
 
118
  each: {g_best, g_best_fp32.onnx, g_best.onnx, config.json}
119
  ```
 
70
  | `monarch_fc` | 2.38 M | 2.843 | 2.831 | +0.012 |
71
  | `monarch_full` | 1.10 M | 2.838 | 2.846 | −0.009 |
72
 
73
+ #### Monarch block-count sweep (`nblocks` 5 / 8 / 10 / 20)
74
+
75
+ Same architecture as `monarch_8` — hidden 400, fc 600, both FCs and both GRU
76
+ projections Monarch — with **only `nblocks` varying**, so parameters move
77
+ without any other change. RTF is int8 on CPU; the three new runs were measured
78
+ together on an idle box.
79
+
80
+ | run | nblocks | params | FP32 PESQ | int8 PESQ | Δ (FP32→int8) | int8 RTF |
81
+ | ------------ | ------: | -----: | --------: | --------: | ------------: | -------: |
82
+ | `monarch_5` | 5 | 0.88 M | 2.852 | 2.858 | −0.007 | 0.017 |
83
+ | `monarch_8` | 8 | 0.55 M | **2.861** | 2.856 | +0.005 | 0.027 |
84
+ | `monarch_10` | 10 | 0.44 M | 2.849 | 2.842 | +0.007 | 0.014 |
85
+ | `monarch_20` | 20 | 0.23 M | 2.849 | 2.854 | −0.005 | 0.013 |
86
+
87
+ **4× parameters, 0.012 PESQ spread** — the 0.23 M `monarch_20` matches the
88
+ 0.88 M `monarch_5`, and int8 stays loss-free (|Δ| ≤ 0.007) even at the smallest
89
+ size. `monarch_20` is the cheapest model in the family. The `monarch_8` RTF was
90
+ measured in an earlier session under different load, so it is not comparable to
91
+ the other three.
92
+
93
  ### Block-diagonal, dense, butterfly
94
 
95
  | run | params | FP32 PESQ | int8 PESQ | Δ (FP32→int8) |
 
114
  NSNet2 predicts a magnitude mask and reuses the noisy phase, which caps PESQ
115
  regardless of how expressive the mask predictor is. **The dense model was
116
  already over-parameterized** — which is exactly why aggressive structuring is
117
+ nearly free. For deployment, take the smallest (`blockdiag_8` / `monarch_20`).
118
+ The block-count sweep extends this: 4× parameters across `nblocks` 5→20 moves
119
+ PESQ by 0.012, within run-to-run noise.
120
  - **Genuine Monarch beats block-diagonal, but marginally** (+0.011…+0.038 FP32 at
121
  matched `nblocks`) and it costs parameters — its second factor makes it larger.
122
  Consistent with the saturation above.
 
135
  baseline/ blockdiag_8/ monarch_8/ butterfly_fc/
136
  blockdiag_fc/ blockdiag_full/ monarch_fc/ butterfly_full/
137
  wide_blockdiag/ monarch_full/ wide_monarch/ butterfly_ortho/
138
+ monarch_5/ monarch_10/ butterfly_2blocks/
139
+ monarch_20/
140
 
141
  each: {g_best, g_best_fp32.onnx, g_best.onnx, config.json}
142
  ```
monarch_10/config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_gpus": 0,
3
+ "batch_size": 256,
4
+ "learning_rate": 0.003,
5
+ "adam_b1": 0.8,
6
+ "adam_b2": 0.99,
7
+ "lr_decay": 0.99,
8
+ "seed": 1234,
9
+ "hidden_dim": 400,
10
+ "fc_hidden_dim": 600,
11
+ "num_gru_layers": 2,
12
+ "compress_factor": 0.3,
13
+ "linear": {
14
+ "kind": "monarch",
15
+ "nblocks": 10
16
+ },
17
+ "gru": {
18
+ "kind": "triton_monarch",
19
+ "nblocks": 10,
20
+ "struct_input": true
21
+ },
22
+ "sampling_rate": 16000,
23
+ "segment_size": 32000,
24
+ "n_fft": 512,
25
+ "hop_size": 256,
26
+ "win_size": 512,
27
+ "num_workers": 8,
28
+ "quant": {
29
+ "enabled": false,
30
+ "n_calib_utts": 200
31
+ },
32
+ "dist_config": {
33
+ "dist_backend": "nccl",
34
+ "dist_url": "tcp://localhost:54321",
35
+ "world_size": 1
36
+ }
37
+ }
monarch_10/g_best ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b358c5c5de3b3ac8de436ff0da76d3f99a5653253069ff29c24fd7b0017f308
3
+ size 1833745
monarch_10/g_best.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d63d2b56ad3b8f18d8a213e890ddee31f3710adb19eb8e6134efb427daaba560
3
+ size 566366
monarch_10/g_best_fp32.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f9a194bc3374b8972b06f28df110417312b3bfa2c72e66e52315bece772b7d3
3
+ size 1805283
monarch_20/config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_gpus": 0,
3
+ "batch_size": 256,
4
+ "learning_rate": 0.003,
5
+ "adam_b1": 0.8,
6
+ "adam_b2": 0.99,
7
+ "lr_decay": 0.99,
8
+ "seed": 1234,
9
+ "hidden_dim": 400,
10
+ "fc_hidden_dim": 600,
11
+ "num_gru_layers": 2,
12
+ "compress_factor": 0.3,
13
+ "linear": {
14
+ "kind": "monarch",
15
+ "nblocks": 20
16
+ },
17
+ "gru": {
18
+ "kind": "triton_monarch",
19
+ "nblocks": 20,
20
+ "struct_input": true
21
+ },
22
+ "sampling_rate": 16000,
23
+ "segment_size": 32000,
24
+ "n_fft": 512,
25
+ "hop_size": 256,
26
+ "win_size": 512,
27
+ "num_workers": 8,
28
+ "quant": {
29
+ "enabled": false,
30
+ "n_calib_utts": 200
31
+ },
32
+ "dist_config": {
33
+ "dist_backend": "nccl",
34
+ "dist_url": "tcp://localhost:54321",
35
+ "world_size": 1
36
+ }
37
+ }
monarch_20/g_best ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c10a754e8361cd88810143b7a811d6cf8328e4120386e14d6f57e9d6d8d661d3
3
+ size 960273
monarch_20/g_best.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8c73b0de34db72a6ce4b820059c9ab0bfd3ebf1dcf465d1a349255cb4fe57d2
3
+ size 345174
monarch_20/g_best_fp32.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ca0f2802c8032fa043bbd19fcbbb59d7c96b4e4579b373296ee024172a6bb0f
3
+ size 931761
monarch_5/config.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "num_gpus": 0,
3
+ "batch_size": 256,
4
+ "learning_rate": 0.003,
5
+ "adam_b1": 0.8,
6
+ "adam_b2": 0.99,
7
+ "lr_decay": 0.99,
8
+ "seed": 1234,
9
+ "hidden_dim": 400,
10
+ "fc_hidden_dim": 600,
11
+ "num_gru_layers": 2,
12
+ "compress_factor": 0.3,
13
+ "linear": {
14
+ "kind": "monarch",
15
+ "nblocks": 5
16
+ },
17
+ "gru": {
18
+ "kind": "triton_monarch",
19
+ "nblocks": 5,
20
+ "struct_input": true
21
+ },
22
+ "sampling_rate": 16000,
23
+ "segment_size": 32000,
24
+ "n_fft": 512,
25
+ "hop_size": 256,
26
+ "win_size": 512,
27
+ "num_workers": 8,
28
+ "quant": {
29
+ "enabled": false,
30
+ "n_calib_utts": 200
31
+ },
32
+ "dist_config": {
33
+ "dist_backend": "nccl",
34
+ "dist_url": "tcp://localhost:54321",
35
+ "world_size": 1
36
+ }
37
+ }
monarch_5/g_best ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69496ae315a21000fd3e471f4e2fda1bce16aff23c7ae2a339eb0ee5cf6eac1c
3
+ size 3580753
monarch_5/g_best.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f04861d3e15f3d87bcaf884de8cafa1e103f289d7bc905d4785d3395e2941e7
3
+ size 1008147
monarch_5/g_best_fp32.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a8b383e6d25a18f71787d74cd8eee264052ff01a0b2f5c52dff899cff175816
3
+ size 3552324