SAME-L decoder LoRA (squeakfix_v3): round-trip artifact fix

README.md

@@ -0,0 +1,253 @@
+---
+language:
+- en
+library_name: stable-audio-3
+license: other
+license_name: stable-audio-community
+license_link: https://huggingface.co/stabilityai/SAME-L/blob/main/LICENSE
+base_model: stabilityai/SAME-L
+tags:
+- music
+- audio
+- autoencoder
+- lora
+- stable-audio-3
+---
+
+# SAME-L Decoder LoRA (`squeakfix_v3`)
+
+A LoRA on **SAME-L's decoder**. It changes how latents are rendered to audio, not
+how latents are produced.
+
+The encoder is untouched, so the latent space, the DiT and every existing DiT
+LoRA stay bit-compatible. This adapter stacks with them in any order.
+
+It exists to fix one specific thing: the high-frequency squeaks and chirps that
+SAME's autoencoder manufactures on percussion when audio goes round the
+encode/decode loop more than once — which is what every chained continuation
+does.
+
+**Trained and used by [the collabage patch](https://huggingface.co/thepatch).**
+In production in a live `stable-audio-3` backend since 2026-08-14.
+
+---
+
+## Usage
+
+Requires the `target: "decoder"` support in
+[Stability-AI/stable-audio-3](https://github.com/Stability-AI/stable-audio-3).
+No special loading path — the `target` field in the checkpoint config is what
+routes it.
+
+```bash
+python run_gradio.py --model medium-base \
+    --lora-ckpt-path squeakfix_v3.safetensors
+```
+
+Stacked with a DiT LoRA, in either order:
+
+```bash
+python run_gradio.py --model medium-base \
+    --lora-ckpt-path my_style.safetensors squeakfix_v3.safetensors
+```
+
+Directly on the autoencoder:
+
+```python
+from stable_audio_3 import AutoencoderModel
+from stable_audio_3.models.lora.loader import load_and_apply_loras
+
+ae = AutoencoderModel.from_pretrained("same-l", device="cuda").autoencoder
+load_and_apply_loras(ae, ["squeakfix_v3.safetensors"], "autoencoder")
+
+latents = ae.encode(waveform)
+audio = ae.decode(latents)
+```
+
+Strength is a **separate call** — `lora_configs` at inference only gates the
+sigma interval and layer filter:
+
+```python
+from stable_audio_3.models.lora.model import set_lora_strength
+set_lora_strength(ae.decoder, 1.0)   # 0.0 renders at the stock decode
+```
+
+Ship at strength **1.0**. It was trained there and it is where it was auditioned.
+
+---
+
+## What it fixes
+
+SAME downsamples 44.1 kHz stereo by 4096×, so there is one latent frame per
+93 ms and a snare attack lives entirely inside a single frame. The decoder does
+not reconstruct that attack — it resynthesizes one, slightly differently every
+pass.
+
+A single generation is fine. The problem is **feedback**: a continuation
+re-encodes previously decoded audio, so N chained continuations put the audio
+through N round trips. Measured on the round-trip ladder with no DiT involved,
+invented HF energy climbs ~15 dB over six iterations while lost HF stays flat,
+and every one of the twelve worst frames lands within 35 ms of a detected onset.
+
+Nothing in SAME's training objective ever asked that `encode(decode(z)) ≈ z` —
+every loss is `real audio → latent → real audio` — so round-trip idempotence has
+no reason to hold, and it does not. This adapter adds that objective after the
+fact, on the decoder only.
+
+### Measured effect
+
+**The win is at re-encode depth.** Every continuation and every transform
+re-encodes previously decoded audio, so a chained session compounds. Measured on
+audio the model generated (medium + a DiT LoRA), the same latents decoded through
+each adapter — HF tonality, i.e. spectral peakiness, which is what "the hi-hat
+became a whistle" looks like numerically:
+
+| | tonal p95 at 1x | after 3 re-encodes | cost of depth |
+|---|---|---|---|
+| stock | 8.09 | 12.80 | **+4.71** |
+| `squeakfix_v3` | 8.29 | 9.06 | **+0.78** |
+
+The adapter cuts degradation-per-re-encode by ~83%. Chained continuations are
+where it earns its place.
+
+**At 1× this metric is flat** (8.29 vs 8.09 — slightly worse, inside the noise).
+A subtle improvement on single generations is reported by ear, and it is
+plausible that it lives in pops and transient smear rather than tonality, but it
+is not something these numbers demonstrate. Treat 1× as neutral and the re-encode
+behaviour as the reason to use this.
+
+On the round-trip ladder against real held-out audio, which is the eval the
+checkpoint was selected against:
+
+| metric | stock | `squeakfix_v3` | Δ |
+|---|---|---|---|
+| tonality p95 | 17.08 | **9.41** | −7.67 |
+| transient excess (dB) | 6.32 | **1.24** | −5.08 |
+| latent drift per round trip | 10.91 % | **5.85 %** | −5.06 pp |
+| onset artifact (dB) | −3.50 | −7.53 | −4.03 |
+
+`transient excess` is added-HF on transient frames minus that on sustained
+frames; positive means the artifact rides percussion, which is its signature.
+
+---
+
+## Honest limits
+
+**Any decoder LoRA trained this way stamps a comb into the output — this one
+doesn't, and that is the whole reason it exists.** The autoencoder's un-patch is
+a bare reshape at `patch_size 256`, so output channel index and
+time-position-within-patch are the same axis, and any channel-wise bias the
+adapter learns is repeated identically in every patch. That is a harmonic comb at
+`sr/256 = 172.27 Hz`, with an F3 fundamental — it reads as a wrong note, not as
+noise. It is inaudible in a mix and obvious once a stem separator strips the
+masking content, which is how it was found.
+
+`squeakfix_v3` was trained with an explicit penalty on that structure
+(`--lambda_patch 30`). A `v1` trained identically *without* it is the control.
+
+Measured on the distribution that matters — audio **generated by the model
+itself** (medium + a DiT LoRA), the same latents decoded through each adapter, so
+the decoder is the only variable. Comb excess in dB over a control comb at
+meaningless spacing, mean of 4 generations:
+
+| variant | comb excess | vs stock |
+|---|---|---|
+| stock | 2.80 | — |
+| v1 @ strength 1 | 13.70 | **+10.90** |
+| **v3 @ strength 1** | 1.97 | **−0.83** |
+| stock, +3 round trips | 1.64 | −1.17 |
+| v1 @ 1, +3 round trips | 13.34 | +10.54 |
+| **v3 @ 1, +3 round trips** | 1.37 | **−1.43** |
+| v1 @ strength 2 | 18.52 | +15.72 |
+| **v3 @ strength 2** | 3.49 | **+0.68** |
+
+v1 adds ~11 dB of comb. **v3 sits at or slightly below stock**, and stays there
+under re-encoding. Even at strength 2 — double what it ships at, where every
+effect is 2–4× clearer — it is within 0.7 dB of stock, while v1 is nearly 16 dB
+above it.
+
+One caveat for anyone reproducing this: on **real** audio pushed round the
+autoencoder, rather than on model output, v3 measures +1.5 to +4.0 dB of comb
+excess instead of ~0. Real music and model output are different distributions and
+the artifact does not behave the same on both. Measure on generated audio — that
+is what the decoder is actually asked to render, and it is where the artifact was
+audible in the first place.
+
+Other things worth knowing:
+
+- **This is one checkpoint from one dataset**, and it was selected partly by ear.
+  The in-loop ladder kept improving past step 2000; step 2000 is what was chosen
+  on re-encode behaviour and listening. Your material may want a different one.
+- **The squeaks it targets are partly an upstream prompt-distribution problem.**
+  Out-of-distribution prompts produce far more of the offending frames than
+  in-distribution ones. This adapter reduces the artifact; it does not make
+  prompt choice irrelevant.
+- **It is a SAME-L adapter.** SAME-S has the same patch grid but its artifact
+  sits in a different band (1–8 kHz rather than 6 kHz and up) and wants a
+  different recipe. Loading this on SAME-S will not error — it will just sound
+  wrong.
+- **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
+  relative, decoder 7e-3 max abs). Any A/B you run against this adapter has to be
+  read as a multiple of those floors; absolute tolerances are meaningless.
+
+---
+
+## Training
+
+| | |
+|---|---|
+| base model | `stabilityai/SAME-L` |
+| target | `decoder` (encoder frozen) |
+| rank / alpha | 16 / 16.0 |
+| trainable | 5.63M params, 98 tensors (~1.30 % of the decoder) |
+| checkpoint | step 2000 of a run configured for 8000, stopped at 4000 |
+| precision | fp32 training, fp16 storage |
+| hardware | single GB10 (DGX Spark), ~2.9 s/step at 10 s crops, under 10 GB |
+
+```bash
+python scripts/train_decoder_lora.py \
+    --model same-l --rank 16 --steps 8000 --lr 1e-4 \
+    --w_real 0.4 --w_drift 0.3 --w_dit 0.3 \
+    --lambda_cycle 10.0 --lambda_tonal 0.3 --lambda_patch 30 \
+    --patch_size 256 --crop_seconds 10 --seed 0 \
+    --dit_latent_dir out/dit_latents \
+    --data_dir /path/to/audio --eval_audio /path/to/holdout.wav \
+    --out_dir out/declora
+```
+
+The objective is self-supervised — audio only, no captions, no paired data.
+Latents come from three buckets: real audio, drifted audio (`E(D(E(x)))` at depth
+≥ 1, targeted against the **original** audio so the decoder is asked to undo
+accumulated drift), and latents sampled from the base DiT. The DiT bucket earns
+its place because generated latents drift 15–18.5 % against real audio's 10.5 %,
+and that gap is in the base model rather than in any adapter.
+
+Losses: multi-resolution log-STFT L1 (K-weighted, kept deliberately — this
+decoder over-generates HF, so removing the perceptual de-emphasis would push the
+wrong way), relative round-trip consistency `‖E(D(z)) − z‖/‖z‖` with the encoder
+frozen, a **one-sided** HF tonality penalty (so legitimate cymbal ring and
+sibilance are never punished), and the patch-grid penalty described above. No
+discriminator: adversarial training makes a decoder invent *more* detail, and the
+failure mode of plain reconstruction losses is the direction this wants.
+
+Full method, ablations and traps: [Decoder LoRA
+workflow](https://github.com/Stability-AI/stable-audio-3/blob/main/docs/workflows/decoder-lora.md).
+
+---
+
+## Files
+
+| file | what it is |
+|---|---|
+| `squeakfix_v3.safetensors` | the adapter — SAME-L decoder, rank 16, trained with `--lambda_patch 30` |
+
+Provenance (base model, step, full recipe) is embedded in the file's
+`lora_config` metadata, so it identifies itself without this card.
+
+The `v1` column in the tables above is an internal checkpoint trained on the
+identical recipe with `--lambda_patch 0`. It is not published — it is reported
+because "the patch penalty matters" is a claim that needs the control.
+
+## Citation
+
+The autoencoder this adapts: [SAME](https://arxiv.org/abs/2605.18613).

squeakfix_v3.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99942d64e822138fc67445d3c664f31c714ca62cf5d9a21b20ba33a41d9e99ed
+size 11278296