thepatch
/

same-l-decoder-lora

@@ -17,82 +17,46 @@ tags:
 # SAME-L Decoder LoRA (`squeakfix_v3`)
 A LoRA on **SAME-L's decoder**. It changes how latents are rendered to audio, not
-how latents are produced — the encoder is untouched, so the latent space, the DiT
-and every existing DiT LoRA stay bit-compatible, and this stacks with them in any
-order.
-It fixes one thing: the high-frequency squeaks SAME's autoencoder manufactures on
-percussion when audio goes round the encode/decode loop more than once. SAME
-downsamples 44.1 kHz stereo by 4096×, so a snare attack lives entirely inside a
-single latent frame — the decoder doesn't reconstruct that attack, it
-resynthesizes one, slightly differently every pass. Nothing in SAME's training
-objective ever asked that `encode(decode(z)) ≈ z`; every loss is
-`real audio → latent → real audio`. This adapter adds that objective after the
-fact, on the decoder only. Since a continuation or a transform re-encodes its
-input, a chained session compounds the artifact — that's where this earns its
-place.
-Trained and used by [the collabage patch](https://huggingface.co/thepatch), in
-production in a live `stable-audio-3` backend since 2026-08-14. Ship at
-strength **1.0** — that's where it was trained and auditioned.
-## Hear it
-Since we keep struggling to produce clean metrics surrounding this artifact, we
-place these examples so you can really hear what it's doing.
-Same latent, same seed, decoder adapter the only variable; every clip is 15 s,
-peak-matched to −1 dBFS so level can't masquerade as quality. Generated with the
-`koan` DiT LoRA on `medium`.
-### Round trips — where it earns its place
 | | stock decoder | `squeakfix_v3` |
 |---|---|---|
-| **jungle** — fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_v3_d0.wav"></audio> |
-| **jungle** — after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_v3_d3.wav"></audio> |
-| **drums** — fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_v3_d0.wav"></audio> |
-| **drums** — after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_v3_d3.wav"></audio> |
-| **neurofunk** — fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d0.wav"></audio> |
-| **neurofunk** — after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d3.wav"></audio> |
-These three prompts were **not** selected for artifact content — they are
-ordinary koan generations, and the fresh-generation rows are here so you can
-follow the progression rather than because much is audible in them. Three round
-trips is a short chained session: two continuations and a transform get you
-there. jungle is the worst of the three; drums and neurofunk are unremarkable.
-It isn't winning by sanding the top end, which is the easy way to fake this. At
-d3 all three hold **2.0 to 9.3 dB more** 12–16 kHz and 16–22 kHz energy than the
-stock render, at a higher RMS, while showing a fraction of the tonal frames. The
-only band that ever drops is 8–12 kHz on the drums clip, by 0.9 dB — and that is
-exactly where the artifact concentrates.
-### A fresh generation — subtler, but we do hear it
-We swear we hear it on initial generations too, and it is **much** more subtle.
-This pair is the worst of 16 candidates, screened by how far tonality runs away
-over 3 round trips with the adapter off — screening is how you find an audible
-A/B for something this small, and it means this pair is a worst case, not a
-typical one.
 | | stock decoder | `squeakfix_v3` |
 |---|---|---|
-| **glitch hop** — fresh generation, screened | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/glitchhop_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/glitchhop_v3_d0.wav"></audio> |
-What changes here is **air, not squeak count**. Across the three screened
-generations the adapter moves tonality by −0.37 dB at d0 — nothing — while
-restoring **1.5 to 2.4 dB** of 16–22 kHz top octave in every one. On this
-particular pair tonality moves −0.08 dB, i.e. not at all, and air moves +2.4 dB.
-If you are looking for the difference in a tonal-frame count you will not find
-it.
 ## Usage
 Requires `target: "decoder"` support in
 [Stability-AI/stable-audio-3](https://github.com/Stability-AI/stable-audio-3).
-The `target` field in the checkpoint config is what routes it; there's no special
-loading path.
 ```bash
 python run_gradio.py --model medium-base --lora-ckpt-path squeakfix_v3.safetensors
@@ -114,79 +78,68 @@ set_lora_strength(ae.decoder, 1.0)   # separate call; 0.0 is the stock decode
 audio = ae.decode(ae.encode(waveform))
 ```
-## Measured
-Four `koan` generations of 2 minutes, screened from 12 candidates by how far
-6–16 kHz tonality runs away with the adapter off. Each latent is decoded twice —
-adapter off, adapter on — and each chain re-encodes in place, so dN is genuinely
-N round trips of that decoder config. The adapter is the only variable.
-| depth | tonality p95 | tonal frames | 12–16 kHz | SI-SDR vs own d0 |
 |---|---|---|---|---|
-| d0 — fresh generation | 11.25 → 11.33 | 780 → 933 | 20.7 → 21.4 | — |
-| d1 — one continuation or transform | 12.46 → **11.59** | 972 → 1074 | 19.9 → **21.2** | 13.5 → **17.5** |
 | d2 | 14.42 → **11.81** | 1492 → **1246** | 19.3 → **21.0** | 8.9 → **12.8** |
 | d3 | 17.03 → **12.06** | 2453 → **1411** | 18.6 → **20.9** | 6.7 → **10.4** |
-**Stock tonality climbs +5.79 dB over three round trips. With the adapter it
-climbs +0.73 dB.** That holds on every generation individually — stock +4.98 to
-+6.97, the adapter +0.18 to +1.40 — and it is the whole claim: the artifact
-stops compounding.
-**The tonal-frame column rises at d0 and d1, and that is not a regression.**
-That count rewards peakiness, and the adapter puts high-frequency energy back;
-restored harmonics read as tonal. It is also dominated by two of the four
-generations, which start near 1000 and 2100 frames because glitchcore genuinely
-is tonal. The other two start at **zero** and stay there with the adapter — 0 →
-39 and 0 → 8 at d3 — while stock reaches 1038 and 268. Where the artifact has
-room to appear, the adapter suppresses it; where the material is already tonal,
-the count cannot tell artifact from content. Read p95, not the count.
-**It is not sanding the top end**, which is the easy way to post a good number.
-At every depth it holds more 12–16 kHz energy than stock and the gap *widens*
-with depth (+0.70, +1.31, +1.76, +2.30 dB at d0–d3), while SI-SDR against its
-own d0 runs 3.6 to 3.9 dB better. It drifts less rather than erasing what drifts.
-At d0 the change is air, not squeak count: on a separate screened 30 s set the
-adapter moves tonality by −0.37 dB — nothing — while restoring 1.5 to 2.4 dB of
-16–22 kHz top octave in all three generations. Trust the clips above over any of
-these numbers.
 This table is `koan` at 120 s and fp16. An earlier unscreened run covering `kev`
-as well, at both 30 s and 120 s, showed the same direction at depth.
 ## Limits
-- **Decoder LoRAs trained this way stamp a comb into the output — this one
-  doesn't, and that's most of why it exists.** The un-patch is a bare reshape at
-  `patch_size 256`, so output channel and time-position-within-patch are the same
-  axis, and any channel-wise bias the adapter learns repeats identically in every
-  patch: a harmonic comb at `sr/256 = 172.27 Hz` that reads as a wrong note, not
-  as noise. Inaudible in a mix, obvious once a stem separator strips the masking
-  content, which is how it was found. `--lambda_patch 30` is what prevents it. On
-  model output, a `v1` control trained on the identical recipe with
-  `--lambda_patch 0` sits **+10.9 dB** of comb excess over stock; v3 sits
-  **−0.8 dB**, and both hold that gap through three further round trips. On
-  *real* audio rather than model output v3 measures +1.5 to +4.0 dB — different
-  distribution, so measure on generated audio.
-- **SAME-L only.** SAME-S shares the patch grid but its artifact sits 1–8 kHz and
-  wants a different recipe. Loading this there won't error; it'll sound wrong.
-- **The gain scales with re-encode depth** — small for one-shot generation, it
-  compounds across chained continuations.
-- **The squeaks are partly an upstream prompt-distribution problem.** OOD prompts
-  produce far more offending frames; this reduces the artifact, it doesn't make
-  prompt choice irrelevant.
-- **One checkpoint, one dataset, chosen partly by ear** at step 2000 on re-encode
-  behaviour. Your material may want a different one.
 - **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
-  relative, decoder 7e-3 max abs) — read any A/B as a multiple of those floors.
 ## Training
-Self-supervised — audio only, no captions, no paired data. Rank 16 / alpha 16,
-5.63M trainable params (~1.30 % of the decoder), step 2000 of a run configured
-for 8000, fp32 training and fp16 storage, single GB10 (DGX Spark) at ~2.9 s/step
-on 10 s crops, under 10 GB.
 ```bash
 python scripts/train_decoder_lora.py \
@@ -200,15 +153,15 @@ python scripts/train_decoder_lora.py \
 ```
 Latents come from three buckets: real audio, drifted audio (`E(D(E(x)))` at depth
-≥ 1, targeted against the **original** so the decoder is asked to undo
-accumulated drift), and latents sampled from the base DiT. No discriminator —
-adversarial training makes a decoder invent *more* detail, and the failure mode
-of plain reconstruction losses is the direction this wants.
 Full method, losses, ablations and traps: [Decoder LoRA
 workflow](https://github.com/Stability-AI/stable-audio-3/blob/main/docs/workflows/decoder-lora.md).
-Provenance — base model, step, full recipe — is embedded in the file's
-`lora_config` metadata, so the checkpoint identifies itself without this card.
-`v1` (identical recipe, `--lambda_patch 0`) is an internal control, not published.
-The autoencoder this adapts: [SAME](https://arxiv.org/abs/2605.18613).

 # SAME-L Decoder LoRA (`squeakfix_v3`)
 A LoRA on **SAME-L's decoder**. It changes how latents are rendered to audio, not
+how latents are produced, so the latent space, the DiT and every existing DiT
+LoRA stay bit-compatible and this stacks with them in any order.
+It targets the high-frequency squeaks SAME's autoencoder adds to percussion when
+audio goes round the encode/decode loop more than once. A continuation or a
+transform re-encodes its input, so a chained session compounds them.
+## Listen
 | | stock decoder | `squeakfix_v3` |
 |---|---|---|
+| **jungle**, fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_v3_d0.wav"></audio> |
+| **jungle**, after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/jungle_v3_d3.wav"></audio> |
+| **drums**, fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_v3_d0.wav"></audio> |
+| **drums**, after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/drums_v3_d3.wav"></audio> |
+| **neurofunk**, fresh generation | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d0.wav"></audio> |
+| **neurofunk**, after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d3.wav"></audio> |
+15 s clips, same latent and seed, decoder adapter the only variable, peak-matched
+to −1 dBFS. Generated with the `koan` DiT LoRA on `medium`. These three prompts
+were not picked for artifact content, and the fresh-generation rows are there to
+show the progression rather than because much is audible in them.
+### On a fresh generation
 | | stock decoder | `squeakfix_v3` |
 |---|---|---|
+| **glitch hop**, worst of 16 screened | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/glitchhop_stock_d0.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/glitchhop_v3_d0.wav"></audio> |
+Much subtler, and it took screening 16 candidates by tonality runaway to find a
+pair where it carries. What changes here is air rather than squeak count: on this
+pair tonality moves 0.08 dB, which is nothing, while 16-22 kHz comes back
++2.4 dB.
 ## Usage
 Requires `target: "decoder"` support in
 [Stability-AI/stable-audio-3](https://github.com/Stability-AI/stable-audio-3).
+The `target` field in the checkpoint config routes it; there is no special
+loading path. Ship at strength **1.0**, where it was trained and auditioned.
 ```bash
 python run_gradio.py --model medium-base --lora-ckpt-path squeakfix_v3.safetensors
 audio = ae.decode(ae.encode(waveform))
 ```
+## Numbers
+Four `koan` generations of 2 minutes, screened from 12 candidates by tonality
+runaway. Each latent is decoded twice, adapter off and on, and each chain
+re-encodes in place, so dN is N round trips of that decoder config.
+| depth | tonality p95 | tonal frames | 12-16 kHz | SI-SDR vs own d0 |
 |---|---|---|---|---|
+| d0, fresh generation | 11.25 → 11.33 | 780 → 933 | 20.7 → 21.4 | |
+| d1, one continuation | 12.46 → **11.59** | 972 → 1074 | 19.9 → **21.2** | 13.5 → **17.5** |
 | d2 | 14.42 → **11.81** | 1492 → **1246** | 19.3 → **21.0** | 8.9 → **12.8** |
 | d3 | 17.03 → **12.06** | 2453 → **1411** | 18.6 → **20.9** | 6.7 → **10.4** |
+Stock tonality climbs **+5.79 dB** over three round trips. With the adapter it
+climbs **+0.73 dB**, and that holds on all four generations individually (stock
++4.98 to +6.97, adapter +0.18 to +1.40). The artifact stops compounding.
+The frame count rises at d0 and d1 because the adapter puts high-frequency energy
+back and the detector counts peakiness, so restored harmonics read as tonal. It
+is also dominated by two of the four generations, which start near 1000 and 2100
+frames because glitchcore genuinely is tonal. The other two start at zero and
+stay near zero with the adapter (0 → 39 and 0 → 8 at d3) while stock reaches 1038
+and 268. Read p95.
+The adapter holds more 12-16 kHz than stock at every depth and the gap widens
+with re-encoding (+0.70, +1.31, +1.76, +2.30 dB at d0 to d3), while SI-SDR
+against its own d0 runs 3.6 to 3.9 dB better. It drifts less rather than erasing
+what drifts.
 This table is `koan` at 120 s and fp16. An earlier unscreened run covering `kev`
+as well, at 30 s and 120 s, showed the same direction at depth.
 ## Limits
+- **Decoder LoRAs trained this way stamp a comb into the output.** The un-patch
+  is a bare reshape at `patch_size 256`, so output channel and
+  time-position-within-patch are the same axis, and any channel-wise bias the
+  adapter learns repeats in every patch. That is a harmonic comb at
+  `sr/256 = 172.27 Hz`, which reads as a wrong note rather than as noise. It is
+  inaudible in a mix and obvious once a stem separator strips the masking
+  content, which is how it was found. `--lambda_patch 30` is what prevents it: a
+  `v1` control trained without it sits **+10.9 dB** of comb excess over stock
+  while v3 sits **0.8 dB below** stock, and both hold through three further round
+  trips. On real audio rather than model output v3 measures +1.5 to +4.0 dB, so
+  measure on generated audio.
+- **SAME-L only.** SAME-S shares the patch grid but its artifact sits 1-8 kHz and
+  wants a different recipe. Loading this there will not error, it will sound
+  wrong.
+- **The gain scales with re-encode depth**, so it is small for one-shot
+  generation and compounds across chained continuations.
+- **The squeaks are partly a prompt-distribution problem.** Out-of-distribution
+  prompts produce far more of the offending frames.
+- **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
+  listening. Your material may want a different one.
 - **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
+  relative, decoder 7e-3 max abs), so read any A/B as a multiple of those floors.
 ## Training
+Self-supervised, audio only, no captions or paired data. Rank 16 / alpha 16,
+5.63M trainable params (~1.30 % of the decoder), fp32 training and fp16 storage,
+single GB10 (DGX Spark) at ~2.9 s/step on 10 s crops, under 10 GB.
 ```bash
 python scripts/train_decoder_lora.py \
 ```
 Latents come from three buckets: real audio, drifted audio (`E(D(E(x)))` at depth
+≥ 1, targeted against the original so the decoder is asked to undo accumulated
+drift), and latents sampled from the base DiT. No discriminator, since
+adversarial training makes a decoder invent *more* detail.
 Full method, losses, ablations and traps: [Decoder LoRA
 workflow](https://github.com/Stability-AI/stable-audio-3/blob/main/docs/workflows/decoder-lora.md).
+Provenance is embedded in the file's `lora_config` metadata. `v1` (identical
+recipe, `--lambda_patch 0`) is an internal control, not published.
+Trained and used by [the collabage patch](https://huggingface.co/thepatch), in
+production in a live `stable-audio-3` backend since 2026-08-14. The autoencoder
+this adapts: [SAME](https://arxiv.org/abs/2605.18613).