Instructions to use thepatch/same-l-decoder-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Stable Audio 3
How to use thepatch/same-l-decoder-lora with Stable Audio 3:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Card: depth curve, and state the low-depth metric gap plainly
Browse files
README.md
CHANGED
|
@@ -94,30 +94,51 @@ every loss is `real audio β latent β real audio` β so round-trip idempoten
|
|
| 94 |
no reason to hold, and it does not. This adapter adds that objective after the
|
| 95 |
fact, on the decoder only.
|
| 96 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
### Measured effect
|
| 98 |
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
audio the model generated (medium + a DiT LoRA), the same latents decoded through
|
| 102 |
-
each adapter β HF tonality, i.e. spectral peakiness, which is what "the hi-hat
|
| 103 |
-
became a whistle" looks like numerically:
|
| 104 |
|
| 105 |
-
| |
|
| 106 |
|---|---|---|---|
|
| 107 |
-
|
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
|
|
|
|
|
|
| 112 |
|
| 113 |
-
|
| 114 |
-
A subtle improvement on single generations is reported by ear, and it is
|
| 115 |
-
plausible that it lives in pops and transient smear rather than tonality, but it
|
| 116 |
-
is not something these numbers demonstrate. Treat 1Γ as neutral and the re-encode
|
| 117 |
-
behaviour as the reason to use this.
|
| 118 |
|
| 119 |
-
On the round-trip ladder against real held-out audio
|
| 120 |
-
|
| 121 |
|
| 122 |
| metric | stock | `squeakfix_v3` | Ξ |
|
| 123 |
|---|---|---|---|
|
|
@@ -182,6 +203,9 @@ Other things worth knowing:
|
|
| 182 |
Out-of-distribution prompts produce far more of the offending frames than
|
| 183 |
in-distribution ones. This adapter reduces the artifact; it does not make
|
| 184 |
prompt choice irrelevant.
|
|
|
|
|
|
|
|
|
|
| 185 |
- **It is a SAME-L adapter.** SAME-S has the same patch grid but its artifact
|
| 186 |
sits in a different band (1β8 kHz rather than 6 kHz and up) and wants a
|
| 187 |
different recipe. Loading this on SAME-S will not error β it will just sound
|
|
|
|
| 94 |
no reason to hold, and it does not. This adapter adds that objective after the
|
| 95 |
fact, on the decoder only.
|
| 96 |
|
| 97 |
+
### What it sounds like
|
| 98 |
+
|
| 99 |
+
Squeaks and chirps on hi hats, snares and percussion fills β a hat turning into a
|
| 100 |
+
little whistle. Present on a single generation and worse every time the audio goes
|
| 101 |
+
back through the autoencoder, which a continuation and a transform both do.
|
| 102 |
+
|
| 103 |
### Measured effect
|
| 104 |
|
| 105 |
+
Tonal frames (>12 dB tonality, 6β16 kHz) on 2-minute generations, mean of 4, the
|
| 106 |
+
decoder adapter the only variable:
|
|
|
|
|
|
|
|
|
|
| 107 |
|
| 108 |
+
| depth | stock | `squeakfix_v3` | |
|
| 109 |
|---|---|---|---|
|
| 110 |
+
| d0 β fresh generation | 363 | 422 | β16% |
|
| 111 |
+
| d1 β one transform or continuation | 431 | 450 | β4% |
|
| 112 |
+
| d2 | 823 | 518 | **37% fewer** |
|
| 113 |
+
| d3 | 1664 | 615 | **63% fewer** |
|
| 114 |
+
|
| 115 |
+
Same shape at fp16 serving precision, on two different DiT LoRAs, at 30 s and
|
| 116 |
+
120 s. Because a transform and a continuation each re-encode their input,
|
| 117 |
+
anything past the first generation already sits at dβ₯1 and a chained session
|
| 118 |
+
climbs from there β that is where this earns its place.
|
| 119 |
+
|
| 120 |
+
### The honest gap
|
| 121 |
+
|
| 122 |
+
**The squeak is clearly audible at d0 β hi hats and snares especially, reported
|
| 123 |
+
by several listeners β and no metric here shows it.** Tried and flat or slightly
|
| 124 |
+
negative at d0: tonality p95 in 6β16 kHz and 1β8 kHz, frame counts over 10/12/15
|
| 125 |
+
dB, onset-locked tonality excess, fp32 and fp16, both LoRAs, both lengths.
|
| 126 |
+
|
| 127 |
+
The likely reason is that the measurement has no headroom at d0: ~640 of ~10,300
|
| 128 |
+
frames already read as tonal before any re-encoding, because cymbals and
|
| 129 |
+
distorted guitar genuinely are tonal, and a few dozen added squeak frames cannot
|
| 130 |
+
move a count that size. Separating artifact from content works at depth but not
|
| 131 |
+
at d0 without a clean reference, which generated audio does not have.
|
| 132 |
|
| 133 |
+
There is also precedent in this project for the tonality detector pointing the
|
| 134 |
+
wrong way β on SAME-S the clip that sounds squeakiest scores lowest. So the d0
|
| 135 |
+
β16% is not evidence the adapter hurts; it is a regime where this instrument does
|
| 136 |
+
not resolve.
|
| 137 |
|
| 138 |
+
Listen before deciding. Do not take the d0 number as the verdict.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
|
| 140 |
+
On the round-trip ladder against real held-out audio β the eval this checkpoint
|
| 141 |
+
was selected against:
|
| 142 |
|
| 143 |
| metric | stock | `squeakfix_v3` | Ξ |
|
| 144 |
|---|---|---|---|
|
|
|
|
| 203 |
Out-of-distribution prompts produce far more of the offending frames than
|
| 204 |
in-distribution ones. This adapter reduces the artifact; it does not make
|
| 205 |
prompt choice irrelevant.
|
| 206 |
+
- **The benefit scales with re-encode depth.** If your workflow is one-shot
|
| 207 |
+
generation and nothing else, the measurable gain is small. If you chain
|
| 208 |
+
continuations or transforms, it compounds in your favour.
|
| 209 |
- **It is a SAME-L adapter.** SAME-S has the same patch grid but its artifact
|
| 210 |
sits in a different band (1β8 kHz rather than 6 kHz and up) and wants a
|
| 211 |
different recipe. Loading this on SAME-S will not error β it will just sound
|