Stable Audio 3
English
music
audio
autoencoder
lora
thecollabagepatch commited on
Commit
a516b73
Β·
verified Β·
1 Parent(s): 6156258

Rebuild Numbers on the base model with descriptive prompts; add the prompt-distribution result

Browse files
Files changed (1) hide show
  1. README.md +33 -34
README.md CHANGED
@@ -89,38 +89,37 @@ audio = ae.decode(ae.encode(waveform))
89
 
90
  ## Numbers
91
 
92
- Four generations of 2 minutes, made with a DiT LoRA on `medium` and screened
93
- from 12 candidates by tonality runaway. Each latent is decoded twice, adapter off and on, and each chain
94
- re-encodes in place, so dN is N round trips of that decoder config.
 
95
 
96
- | depth | tonality p95 | tonal frames | 12-16 kHz | SI-SDR vs own d0 |
97
  |---|---|---|---|---|
98
- | d0, fresh generation | 11.25 β†’ 11.33 | 780 β†’ 933 | 20.7 β†’ 21.4 | |
99
- | d1, one continuation | 12.46 β†’ **11.59** | 972 β†’ 1074 | 19.9 β†’ **21.2** | 13.5 β†’ **17.5** |
100
- | d2 | 14.42 β†’ **11.81** | 1492 β†’ **1246** | 19.3 β†’ **21.0** | 8.9 β†’ **12.8** |
101
- | d3 | 17.03 β†’ **12.06** | 2453 β†’ **1411** | 18.6 β†’ **20.9** | 6.7 β†’ **10.4** |
102
-
103
- Stock tonality climbs **+5.79 dB** over three round trips. With the adapter it
104
- climbs **+0.73 dB**, and that holds on all four generations individually (stock
105
- +4.98 to +6.97, adapter +0.18 to +1.40). The artifact stops compounding.
106
-
107
- The frame count rises at d0 and d1 because the adapter puts high-frequency energy
108
- back and the detector counts peakiness, so restored harmonics read as tonal. It
109
- is also dominated by two of the four generations, which start near 1000 and 2100
110
- frames because glitchcore genuinely is tonal. The other two start at zero and
111
- stay near zero with the adapter (0 β†’ 39 and 0 β†’ 8 at d3) while stock reaches 1038
112
- and 268. Read p95.
113
-
114
- The adapter holds more 12-16 kHz than stock at every depth and the gap widens
115
- with re-encoding (+0.70, +1.31, +1.76, +2.30 dB at d0 to d3), while SI-SDR
116
- against its own d0 runs 3.6 to 3.9 dB better. It drifts less rather than erasing
117
- what drifts.
118
-
119
- Run again on the base model with no DiT LoRA, the same ladder gives a stock climb
120
- of +6.73 dB against the adapter's +1.81 dB, so this is the autoencoder's
121
- behaviour rather than something a LoRA introduces. An earlier unscreened run
122
- covering a second, quite different LoRA, at 30 s and 120 s, showed the same
123
- direction at depth. All of these are fp16 at 120 s.
124
 
125
  ## Limits
126
 
@@ -140,10 +139,10 @@ direction at depth. All of these are fp16 at 120 s.
140
  wrong.
141
  - **The gain scales with re-encode depth**, so it is small for one-shot
142
  generation and compounds across chained continuations.
143
- - **The squeaks are worst on out-of-distribution prompts**, which is easy to hit
144
- when a DiT LoRA has pulled the model toward a narrow style. We have not
145
- measured with-LoRA against without-LoRA on matched content, and on its own the
146
- base model runs away slightly more rather than less.
147
  - **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
148
  listening. Your material may want a different one.
149
  - **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
 
89
 
90
  ## Numbers
91
 
92
+ Four base-model generations of 2 minutes on ordinary descriptive prompts,
93
+ screened from 12 candidates by tonality runaway. Each latent is decoded twice,
94
+ adapter off and on, and each chain re-encodes in place, so dN is N round trips of
95
+ that decoder config. The adapter is the only variable.
96
 
97
+ | depth | tonality p95 | tonal frames | 16-22 kHz | SI-SDR vs own d0 |
98
  |---|---|---|---|---|
99
+ | d0, fresh generation | 9.80 β†’ **9.10** | 87 β†’ **62** | 3.16 β†’ **4.34** | |
100
+ | d1, one continuation | 11.82 β†’ **9.73** | 462 β†’ **64** | 0.95 β†’ **3.76** | 16.9 β†’ **21.0** |
101
+ | d2 | 14.79 β†’ **10.61** | 2111 β†’ **133** | βˆ’1.36 β†’ **3.32** | 12.4 β†’ **16.2** |
102
+ | d3 | 17.85 β†’ **11.74** | 4279 β†’ **453** | βˆ’3.36 β†’ **2.96** | 10.1 β†’ **13.6** |
103
+
104
+ The adapter is ahead at every depth, including a fresh generation. Stock tonality
105
+ climbs +8.05 dB over three round trips against the adapter's +2.64, and by d3
106
+ stock carries 4279 tonal frames against 453.
107
+
108
+ It is not doing this by sanding the top end. After three round trips the
109
+ adapter's 16-22 kHz sits 0.2 dB from a **fresh** stock render, at unchanged RMS,
110
+ while stock has lost 6.5 dB of it. Stock's 12-16 kHz meanwhile *rises* with
111
+ depth: the invented energy is replacing the air.
112
+
113
+ Prompt distribution matters more here than anything else. The same ladder run on
114
+ terse genre-tag prompts, which are out of distribution for the base model, starts
115
+ at **1649** tonal frames instead of 87. With a DiT LoRA attached, which makes
116
+ those tags in-distribution again, it starts at 780. The artifact is worst where
117
+ the prompt sits furthest from what the model expects, and re-encoding compounds
118
+ whatever it starts with.
119
+
120
+ A DiT LoRA does not change the shape: on that run stock climbs +5.79 dB against
121
+ the adapter's +0.73, and the adapter ends d3 holding more 12-16 kHz and 16-22 kHz
122
+ than a fresh stock render. All runs are fp16 at 120 s.
 
 
123
 
124
  ## Limits
125
 
 
139
  wrong.
140
  - **The gain scales with re-encode depth**, so it is small for one-shot
141
  generation and compounds across chained continuations.
142
+ - **The squeaks are worst on out-of-distribution prompts.** Genre-tag prompts on
143
+ the base model start with 19x more tonal frames than descriptive ones. A DiT
144
+ LoRA makes such tags in-distribution again and lands in between. This adapter
145
+ reduces the artifact; it does not make prompt choice irrelevant.
146
  - **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
147
  listening. Your material may want a different one.
148
  - **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %