alexwengg commited on
Commit
677224d
·
verified ·
1 Parent(s): 35a2a3c

Delete ane_fp32

Browse files
Files changed (27) hide show
  1. ane_fp32/README.md +0 -115
  2. ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel +0 -3
  3. ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin +0 -3
  4. ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Manifest.json +0 -18
  5. ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel +0 -3
  6. ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin +0 -3
  7. ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Manifest.json +0 -18
  8. ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel +0 -3
  9. ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin +0 -3
  10. ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Manifest.json +0 -18
  11. ane_fp32/voices/ref_s_1221-135767-0014.bin +0 -3
  12. ane_fp32/voices/ref_s_1789_142896_000022_000005.bin +0 -3
  13. ane_fp32/voices/ref_s_3.bin +0 -3
  14. ane_fp32/voices/ref_s_4.bin +0 -3
  15. ane_fp32/voices/ref_s_4077-13754-0000.bin +0 -3
  16. ane_fp32/voices/ref_s_5.bin +0 -3
  17. ane_fp32/voices/ref_s_5639-40744-0020.bin +0 -3
  18. ane_fp32/voices/ref_s_696_92939_000016_000006.bin +0 -3
  19. ane_fp32/voices/ref_s_908-157963-0027.bin +0 -3
  20. ane_fp32/voices/ref_s_Gavin.bin +0 -3
  21. ane_fp32/voices/ref_s_Nima.bin +0 -3
  22. ane_fp32/voices/ref_s_Vinay.bin +0 -3
  23. ane_fp32/voices/ref_s_Yinghao.bin +0 -3
  24. ane_fp32/voices/ref_s_amused.bin +0 -3
  25. ane_fp32/voices/ref_s_anger.bin +0 -3
  26. ane_fp32/voices/ref_s_disgusted.bin +0 -3
  27. ane_fp32/voices/ref_s_sleepy.bin +0 -3
ane_fp32/README.md DELETED
@@ -1,115 +0,0 @@
1
- ---
2
- license: mit
3
- tags:
4
- - text-to-speech
5
- - styletts2
6
- - coreml
7
- - apple-silicon
8
- - ane
9
- library_name: coreml
10
- pipeline_tag: text-to-speech
11
- ---
12
-
13
- # StyleTTS2 CoreML (LibriTTS) — fp32
14
-
15
- Apple CoreML conversion of [StyleTTS2](https://github.com/yl4579/StyleTTS2)
16
- (LibriTTS checkpoint), split into three fixed-shape mlpackages so each
17
- piece can target the compute unit it runs best on. Targets iOS 17+ /
18
- macOS 14+. fp32 across the board — fp16 / ANE placement is a separate
19
- phase (see Phase 6 in `coreml/PHASE6_FP16_DECODER.md` upstream).
20
-
21
- ## Layout
22
-
23
- ```
24
- styletts2_diffusion_step_256_fp32.mlpackage (~95 MB) one DDIM-style sampler step
25
- styletts2_decoder_noise_2048_fp32.mlpackage (~9.4 MB) HiFi-GAN SineGen + harmonic source
26
- styletts2_decoder_vocoder_2048_fp32.mlpackage (~208 MB) HiFi-GAN body (Snake → cos rewrite)
27
- voices/ pre-extracted ref_s tensors (256-dim, fp32)
28
- ```
29
-
30
- The three mlpackages are designed to be called in this order from a Swift
31
- host: text encoder + BERT + duration encoder run host-side, the diffusion
32
- step is invoked `num_steps` times by the host's ADPM2 sampler, then the
33
- noise + vocoder pair produce 24 kHz audio.
34
-
35
- ## Bucket / shape contract
36
-
37
- | input | shape | notes |
38
- |---|---|---|
39
- | diffusion step | `noise (1, 1, 256)`, `embedding (1, T_tok, 768)`, `sigma (1,)`, `features (1, 256)` | Single fixed `T_tok=256` bucket |
40
- | noise | `F0_curve (1, 4096)`, `noise (1, 1228800, 9)` | `T_mel=2048` bucket; `noise` is fp32 Gaussian |
41
- | vocoder | `asr (1, 512, 2048)`, `F0_curve (1, 4096)`, `N (1, 4096)`, `s (1, 128)`, `har_source (1, 1, 1228800)` | `T_mel=2048` bucket |
42
-
43
- The `T_mel=2048` bucket caps text at ~6.4 s of audio per call (hop=300,
44
- sample_rate=24000). Short utterances pad with zeros up to the bucket on
45
- the host side.
46
-
47
- The noise mlpackage takes `noise` as a graph input (caller-supplied
48
- `torch.randn`); this is the Trial 35 fix that restored fricative
49
- broadband energy. Earlier exports baked a `sin(fn × 100)` substitute for
50
- the upstream `randn_like(sine_waves)` term to keep the trace
51
- deterministic — silent unvoiced regions destroyed /s/, /sh/, /f/.
52
-
53
- ## Digests
54
-
55
- ```
56
- diffusion_step_256: 563ed124a1ddbc3e4fb2374e53ef69966676d1d41bc07b0c25ba9f92a1902af2
57
- decoder_noise_2048: a0b2778f35bbc81316ac5f15fea1e7cf68bf103ce635d590e498b935df7de50b
58
- decoder_vocoder_2048: 9cefdac41e6cc851774bcb21c8aa7ee635a3283d0e658a52f1e3242a6bf1612e
59
- ```
60
-
61
- `package_digest` = `sha256(json([sorted (path, sha256(file)) for file in pkg]))`.
62
- See `scripts/phase5_lock_fp32.py` upstream for the locking workflow.
63
-
64
- ## Voices
65
-
66
- `voices/ref_s_*.bin` — 256-dim `float32` style + prosody embeddings, raw
67
- little-endian (no header). One file per speaker; load with
68
- `Data(contentsOf:)` and copy into a `(1, 256)` MLMultiArray. Filenames
69
- mirror the LibriTTS demo speakers plus a few extras (Yinghao, Vinay,
70
- Nima, Gavin, sleepy, amused, anger, disgusted).
71
-
72
- ## Compute units
73
-
74
- - diffusion step: `cpu_and_gpu` recommended (most ops Snake-rewritten;
75
- ANE placement gated on Phase 6 fp16).
76
- - noise: `cpu_and_gpu` only — fp32 mlprogram cannot run on ANE; in
77
- Phase 6 the SineGen phase accumulator saturates in fp16, so this
78
- stage stays fp32 even after ANE migration.
79
- - vocoder: `cpu_and_gpu` (same fp32 → no ANE rule).
80
-
81
- Setting `compute_units = .cpuAndGPU` from Swift matches how the
82
- conversion harness validates parity.
83
-
84
- ## Conversion source
85
-
86
- Conversion scripts and trial documentation live in the
87
- [mobius](https://github.com/your-org/mobius) repo under
88
- `models/tts/styletts2/`. Key files:
89
-
90
- - `scripts/04_export_diffusion_step_fp32.py` (diffusion step)
91
- - `scripts/04a_export_decoder_noise_fp32.py` (noise — Trial 35)
92
- - `scripts/04b_export_decoder_vocoder_fp32.py` (vocoder — Phase 3b
93
- Snake → cos rewrite)
94
- - `scripts/phase4_chain_parity.py` (PT-vs-CoreML log-mel dB gate;
95
- exit threshold 0.5 dB)
96
- - `scripts/phase5_lock_fp32.py` (digest lock for the three mlpackages)
97
- - `coreml/TRIALS.md` (full trial log including Bug C / Trial 35)
98
- - `coreml/PRECISION.md` (per-stage parity numbers)
99
-
100
- ## License
101
-
102
- MIT. Upstream StyleTTS2 is also MIT-licensed; LibriTTS is CC BY 4.0.
103
- This export reproduces upstream weights without modification — only the
104
- graph layout changes (split + Snake → cos identity rewrite + constfold
105
- SineGen).
106
-
107
- ## Limitations
108
-
109
- - Single fixed bucket per stage; very long inputs require host-side
110
- chunking (not provided here).
111
- - fp32 only. ANE placement requires fp16; see Phase 6 notes upstream.
112
- - LibriTTS-only checkpoint — no LJSpeech / multilingual variants in
113
- this folder.
114
- - Diffusion sampler must run host-side; the mlpackage exposes one step
115
- at a time.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:359949ba9d07ab754eceb53709769d3d7c8390395a6192aa3be5ade2acbea311
3
- size 12838
 
 
 
 
ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:89f32b66bbd82e57b1d6e61e2357780ca771196e1fbbefe5530c1cd05b40acb1
3
- size 9830592
 
 
 
 
ane_fp32/styletts2_decoder_noise_2048_fp32.mlpackage/Manifest.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "fileFormatVersion": "1.0.0",
3
- "itemInfoEntries": {
4
- "5D7D7039-E578-4193-85C9-C2D303886CEE": {
5
- "author": "com.apple.CoreML",
6
- "description": "CoreML Model Specification",
7
- "name": "model.mlmodel",
8
- "path": "com.apple.CoreML/model.mlmodel"
9
- },
10
- "A7A43036-469A-43AC-81C8-38A68C42CDEF": {
11
- "author": "com.apple.CoreML",
12
- "description": "CoreML Model Weights",
13
- "name": "weights",
14
- "path": "com.apple.CoreML/weights"
15
- }
16
- },
17
- "rootModelIdentifier": "5D7D7039-E578-4193-85C9-C2D303886CEE"
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:17ce364dee6a78bd6fce8e7013aa49804b4e5809b2ba68daa120f208312a1329
3
- size 451388
 
 
 
 
ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:d48ca86b5a626c3e68f68128f557c7596210eac833f810bc4ff8fcb198aee581
3
- size 217140224
 
 
 
 
ane_fp32/styletts2_decoder_vocoder_2048_fp32.mlpackage/Manifest.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "fileFormatVersion": "1.0.0",
3
- "itemInfoEntries": {
4
- "3E9BF4B4-76A1-4CCF-8237-C467A80A1DB5": {
5
- "author": "com.apple.CoreML",
6
- "description": "CoreML Model Weights",
7
- "name": "weights",
8
- "path": "com.apple.CoreML/weights"
9
- },
10
- "B2CC0332-5A96-4669-BDE0-AA9E65B676D8": {
11
- "author": "com.apple.CoreML",
12
- "description": "CoreML Model Specification",
13
- "name": "model.mlmodel",
14
- "path": "com.apple.CoreML/model.mlmodel"
15
- }
16
- },
17
- "rootModelIdentifier": "B2CC0332-5A96-4669-BDE0-AA9E65B676D8"
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodel DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:ebea84b22ce6bb729d91a9fa7e8cc047b771b8785f002252540a7ec17714fdcc
3
- size 46991
 
 
 
 
ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:acf1641c418eade9464e8f6ab04b986724865ecc8078c2a066bf50e78802a826
3
- size 99744384
 
 
 
 
ane_fp32/styletts2_diffusion_step_256_fp32.mlpackage/Manifest.json DELETED
@@ -1,18 +0,0 @@
1
- {
2
- "fileFormatVersion": "1.0.0",
3
- "itemInfoEntries": {
4
- "19AC8D29-26A3-453B-B45C-6CFB76203362": {
5
- "author": "com.apple.CoreML",
6
- "description": "CoreML Model Specification",
7
- "name": "model.mlmodel",
8
- "path": "com.apple.CoreML/model.mlmodel"
9
- },
10
- "7F8A3CAC-B058-450A-B06C-CF827933236E": {
11
- "author": "com.apple.CoreML",
12
- "description": "CoreML Model Weights",
13
- "name": "weights",
14
- "path": "com.apple.CoreML/weights"
15
- }
16
- },
17
- "rootModelIdentifier": "19AC8D29-26A3-453B-B45C-6CFB76203362"
18
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ane_fp32/voices/ref_s_1221-135767-0014.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:7c8f34b18352203a90acc1620a8e6564cd76a76efaa3ca9f76c2e7f981925195
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_1789_142896_000022_000005.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:c0ae950fe25ce330dec76e2a312c7e9f08b3770f0c242ec39658c576173b2c53
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_3.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f5920cc5023efd2d0e65bfb1ce8a3ccaf505d00287401182685622cec62912a
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_4.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:87b0125f709791b738f176f7d3d66db1278d12fd092ff71ce7603955907e70a7
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_4077-13754-0000.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:1bc6b4684b7dd023ab557014cf589931df7b6d6eb0fb45b2c9109e5a0b07b251
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_5.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:09ac2a9de005df565e72236e4afc6a46facf5e0e9eaf143ca62b1d14e0edd5b3
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_5639-40744-0020.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:cc121e3357ca8e00bd5ef42f48c5b57c441ae0497aede091654ffae9a7fad93a
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_696_92939_000016_000006.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:44a37a22ab2ffb1af297c9b1714e740ae9c481b3ee20afcac4e9fa8ed82185f6
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_908-157963-0027.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:04026b56d755b3511f40c715465409a0f3fbbd3372de983db5b1b46a532dfc43
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_Gavin.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:2cd47901c6f8f83da48049fcf376480eb6c63b43e6f8359072f66c8fd6c3a9de
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_Nima.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bc32e61a64bb8f5031d619902e60cf99113bd6d90abafbd3962b30c45a9aa1d4
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_Vinay.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:3c87703b18df2959e45e1df350e14640fa4be67397f05a716e80c80a8b8f2a13
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_Yinghao.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b227bd8bb8e359150e43acef549720e7a8820c3afffe713c97707020e49e491b
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_amused.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:58752cec26e778fc643d2b6b425ac7f6b88f466dfc7c90d5e27146c3e5aef03c
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_anger.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:bcda68be1f0016b0e8603cc804dfd18af525f533c398aef8221167688190722c
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_disgusted.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:865912a9750e5d968b5c08606ea8beaf77f26f0f90a4a8ebf4ad105bf4596e97
3
- size 1024
 
 
 
 
ane_fp32/voices/ref_s_sleepy.bin DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:cb9e60a4fb5a9fd5a9e615bbef0b9f60d52ed0be3f23d1e0c8f56e8fb7f751f2
3
- size 1024