Stable Audio 3
English
music
audio
autoencoder
lora
thecollabagepatch commited on
Commit
854d676
Β·
verified Β·
1 Parent(s): d37c273

Card: depth curve, and state the low-depth metric gap plainly

Browse files
Files changed (1) hide show
  1. README.md +41 -17
README.md CHANGED
@@ -94,30 +94,51 @@ every loss is `real audio β†’ latent β†’ real audio` β€” so round-trip idempoten
94
  no reason to hold, and it does not. This adapter adds that objective after the
95
  fact, on the decoder only.
96
 
 
 
 
 
 
 
97
  ### Measured effect
98
 
99
- **The win is at re-encode depth.** Every continuation and every transform
100
- re-encodes previously decoded audio, so a chained session compounds. Measured on
101
- audio the model generated (medium + a DiT LoRA), the same latents decoded through
102
- each adapter β€” HF tonality, i.e. spectral peakiness, which is what "the hi-hat
103
- became a whistle" looks like numerically:
104
 
105
- | | tonal p95 at 1x | after 3 re-encodes | cost of depth |
106
  |---|---|---|---|
107
- | stock | 8.09 | 12.80 | **+4.71** |
108
- | `squeakfix_v3` | 8.29 | 9.06 | **+0.78** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
109
 
110
- The adapter cuts degradation-per-re-encode by ~83%. Chained continuations are
111
- where it earns its place.
 
 
112
 
113
- **At 1Γ— this metric is flat** (8.29 vs 8.09 β€” slightly worse, inside the noise).
114
- A subtle improvement on single generations is reported by ear, and it is
115
- plausible that it lives in pops and transient smear rather than tonality, but it
116
- is not something these numbers demonstrate. Treat 1Γ— as neutral and the re-encode
117
- behaviour as the reason to use this.
118
 
119
- On the round-trip ladder against real held-out audio, which is the eval the
120
- checkpoint was selected against:
121
 
122
  | metric | stock | `squeakfix_v3` | Ξ” |
123
  |---|---|---|---|
@@ -182,6 +203,9 @@ Other things worth knowing:
182
  Out-of-distribution prompts produce far more of the offending frames than
183
  in-distribution ones. This adapter reduces the artifact; it does not make
184
  prompt choice irrelevant.
 
 
 
185
  - **It is a SAME-L adapter.** SAME-S has the same patch grid but its artifact
186
  sits in a different band (1–8 kHz rather than 6 kHz and up) and wants a
187
  different recipe. Loading this on SAME-S will not error β€” it will just sound
 
94
  no reason to hold, and it does not. This adapter adds that objective after the
95
  fact, on the decoder only.
96
 
97
+ ### What it sounds like
98
+
99
+ Squeaks and chirps on hi hats, snares and percussion fills β€” a hat turning into a
100
+ little whistle. Present on a single generation and worse every time the audio goes
101
+ back through the autoencoder, which a continuation and a transform both do.
102
+
103
  ### Measured effect
104
 
105
+ Tonal frames (>12 dB tonality, 6–16 kHz) on 2-minute generations, mean of 4, the
106
+ decoder adapter the only variable:
 
 
 
107
 
108
+ | depth | stock | `squeakfix_v3` | |
109
  |---|---|---|---|
110
+ | d0 β€” fresh generation | 363 | 422 | βˆ’16% |
111
+ | d1 β€” one transform or continuation | 431 | 450 | βˆ’4% |
112
+ | d2 | 823 | 518 | **37% fewer** |
113
+ | d3 | 1664 | 615 | **63% fewer** |
114
+
115
+ Same shape at fp16 serving precision, on two different DiT LoRAs, at 30 s and
116
+ 120 s. Because a transform and a continuation each re-encode their input,
117
+ anything past the first generation already sits at dβ‰₯1 and a chained session
118
+ climbs from there β€” that is where this earns its place.
119
+
120
+ ### The honest gap
121
+
122
+ **The squeak is clearly audible at d0 β€” hi hats and snares especially, reported
123
+ by several listeners β€” and no metric here shows it.** Tried and flat or slightly
124
+ negative at d0: tonality p95 in 6–16 kHz and 1–8 kHz, frame counts over 10/12/15
125
+ dB, onset-locked tonality excess, fp32 and fp16, both LoRAs, both lengths.
126
+
127
+ The likely reason is that the measurement has no headroom at d0: ~640 of ~10,300
128
+ frames already read as tonal before any re-encoding, because cymbals and
129
+ distorted guitar genuinely are tonal, and a few dozen added squeak frames cannot
130
+ move a count that size. Separating artifact from content works at depth but not
131
+ at d0 without a clean reference, which generated audio does not have.
132
 
133
+ There is also precedent in this project for the tonality detector pointing the
134
+ wrong way β€” on SAME-S the clip that sounds squeakiest scores lowest. So the d0
135
+ βˆ’16% is not evidence the adapter hurts; it is a regime where this instrument does
136
+ not resolve.
137
 
138
+ Listen before deciding. Do not take the d0 number as the verdict.
 
 
 
 
139
 
140
+ On the round-trip ladder against real held-out audio β€” the eval this checkpoint
141
+ was selected against:
142
 
143
  | metric | stock | `squeakfix_v3` | Ξ” |
144
  |---|---|---|---|
 
203
  Out-of-distribution prompts produce far more of the offending frames than
204
  in-distribution ones. This adapter reduces the artifact; it does not make
205
  prompt choice irrelevant.
206
+ - **The benefit scales with re-encode depth.** If your workflow is one-shot
207
+ generation and nothing else, the measurable gain is small. If you chain
208
+ continuations or transforms, it compounds in your favour.
209
  - **It is a SAME-L adapter.** SAME-S has the same patch grid but its artifact
210
  sits in a different band (1–8 kHz rather than 6 kHz and up) and wants a
211
  different recipe. Loading this on SAME-S will not error β€” it will just sound