Instructions to use thepatch/same-l-decoder-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Stable Audio 3
How to use thepatch/same-l-decoder-lora with Stable Audio 3:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Point links at the fork branch, add origin note, add base-model ladder
Browse files
README.md
CHANGED
|
@@ -36,7 +36,7 @@ transform re-encodes its input, so a chained session compounds them.
|
|
| 36 |
| **neurofunk**, after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d3.wav"></audio> |
|
| 37 |
|
| 38 |
15 s clips, same latent and seed, decoder adapter the only variable, peak-matched
|
| 39 |
-
to −1 dBFS
|
| 40 |
were not picked for artifact content, and the fresh-generation rows are there to
|
| 41 |
show the progression rather than because much is audible in them.
|
| 42 |
|
|
@@ -51,12 +51,21 @@ pair where it carries. What changes here is air rather than squeak count: on thi
|
|
| 51 |
pair tonality moves 0.08 dB, which is nothing, while 16-22 kHz comes back
|
| 52 |
+2.4 dB.
|
| 53 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
## Usage
|
| 55 |
|
| 56 |
-
Requires `target: "decoder"` support
|
| 57 |
-
[
|
| 58 |
-
|
| 59 |
-
|
|
|
|
| 60 |
|
| 61 |
```bash
|
| 62 |
python run_gradio.py --model medium-base --lora-ckpt-path squeakfix_v3.safetensors
|
|
@@ -80,8 +89,8 @@ audio = ae.decode(ae.encode(waveform))
|
|
| 80 |
|
| 81 |
## Numbers
|
| 82 |
|
| 83 |
-
Four
|
| 84 |
-
runaway. Each latent is decoded twice, adapter off and on, and each chain
|
| 85 |
re-encodes in place, so dN is N round trips of that decoder config.
|
| 86 |
|
| 87 |
| depth | tonality p95 | tonal frames | 12-16 kHz | SI-SDR vs own d0 |
|
|
@@ -107,8 +116,11 @@ with re-encoding (+0.70, +1.31, +1.76, +2.30 dB at d0 to d3), while SI-SDR
|
|
| 107 |
against its own d0 runs 3.6 to 3.9 dB better. It drifts less rather than erasing
|
| 108 |
what drifts.
|
| 109 |
|
| 110 |
-
|
| 111 |
-
|
|
|
|
|
|
|
|
|
|
| 112 |
|
| 113 |
## Limits
|
| 114 |
|
|
@@ -128,8 +140,10 @@ as well, at 30 s and 120 s, showed the same direction at depth.
|
|
| 128 |
wrong.
|
| 129 |
- **The gain scales with re-encode depth**, so it is small for one-shot
|
| 130 |
generation and compounds across chained continuations.
|
| 131 |
-
- **The squeaks are
|
| 132 |
-
|
|
|
|
|
|
|
| 133 |
- **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
|
| 134 |
listening. Your material may want a different one.
|
| 135 |
- **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
|
|
@@ -158,7 +172,7 @@ drift), and latents sampled from the base DiT. No discriminator, since
|
|
| 158 |
adversarial training makes a decoder invent *more* detail.
|
| 159 |
|
| 160 |
Full method, losses, ablations and traps: [Decoder LoRA
|
| 161 |
-
workflow](https://github.com/
|
| 162 |
Provenance is embedded in the file's `lora_config` metadata. `v1` (identical
|
| 163 |
recipe, `--lambda_patch 0`) is an internal control, not published.
|
| 164 |
|
|
|
|
| 36 |
| **neurofunk**, after 3 round trips | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_stock_d3.wav"></audio> | <audio controls src="https://huggingface.co/thepatch/same-l-decoder-lora/resolve/main/samples/neurofunk_v3_d3.wav"></audio> |
|
| 37 |
|
| 38 |
15 s clips, same latent and seed, decoder adapter the only variable, peak-matched
|
| 39 |
+
to −1 dBFS, generated on `medium` with a DiT LoRA attached. These three prompts
|
| 40 |
were not picked for artifact content, and the fresh-generation rows are there to
|
| 41 |
show the progression rather than because much is audible in them.
|
| 42 |
|
|
|
|
| 51 |
pair tonality moves 0.08 dB, which is nothing, while 16-22 kHz comes back
|
| 52 |
+2.4 dB.
|
| 53 |
|
| 54 |
+
## How this came about
|
| 55 |
+
|
| 56 |
+
I do a lot of transformations and continuations using my guitar as input audio.
|
| 57 |
+
When I first noticed these squeaks in the outputs, I mentioned that my trick was
|
| 58 |
+
to stem separate in Ableton and then apply the erosion effect to the percussion
|
| 59 |
+
layer. Claude used that as a guide for refining the methodology. The technical
|
| 60 |
+
version is below.
|
| 61 |
+
|
| 62 |
## Usage
|
| 63 |
|
| 64 |
+
Requires the `target: "decoder"` support on the
|
| 65 |
+
[`decoder-lora` branch of betweentwomidnights/stable-audio-3](https://github.com/betweentwomidnights/stable-audio-3/tree/decoder-lora).
|
| 66 |
+
It is not upstream yet, so that branch is currently the only place a decoder LoRA
|
| 67 |
+
will load. The `target` field in the checkpoint config routes it; there is no
|
| 68 |
+
special loading path. Ship at strength **1.0**, where it was trained and auditioned.
|
| 69 |
|
| 70 |
```bash
|
| 71 |
python run_gradio.py --model medium-base --lora-ckpt-path squeakfix_v3.safetensors
|
|
|
|
| 89 |
|
| 90 |
## Numbers
|
| 91 |
|
| 92 |
+
Four generations of 2 minutes, made with a DiT LoRA on `medium` and screened
|
| 93 |
+
from 12 candidates by tonality runaway. Each latent is decoded twice, adapter off and on, and each chain
|
| 94 |
re-encodes in place, so dN is N round trips of that decoder config.
|
| 95 |
|
| 96 |
| depth | tonality p95 | tonal frames | 12-16 kHz | SI-SDR vs own d0 |
|
|
|
|
| 116 |
against its own d0 runs 3.6 to 3.9 dB better. It drifts less rather than erasing
|
| 117 |
what drifts.
|
| 118 |
|
| 119 |
+
Run again on the base model with no DiT LoRA, the same ladder gives a stock climb
|
| 120 |
+
of +6.73 dB against the adapter's +1.81 dB, so this is the autoencoder's
|
| 121 |
+
behaviour rather than something a LoRA introduces. An earlier unscreened run
|
| 122 |
+
covering a second, quite different LoRA, at 30 s and 120 s, showed the same
|
| 123 |
+
direction at depth. All of these are fp16 at 120 s.
|
| 124 |
|
| 125 |
## Limits
|
| 126 |
|
|
|
|
| 140 |
wrong.
|
| 141 |
- **The gain scales with re-encode depth**, so it is small for one-shot
|
| 142 |
generation and compounds across chained continuations.
|
| 143 |
+
- **The squeaks are worst on out-of-distribution prompts**, which is easy to hit
|
| 144 |
+
when a DiT LoRA has pulled the model toward a narrow style. We have not
|
| 145 |
+
measured with-LoRA against without-LoRA on matched content, and on its own the
|
| 146 |
+
base model runs away slightly more rather than less.
|
| 147 |
- **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
|
| 148 |
listening. Your material may want a different one.
|
| 149 |
- **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
|
|
|
|
| 172 |
adversarial training makes a decoder invent *more* detail.
|
| 173 |
|
| 174 |
Full method, losses, ablations and traps: [Decoder LoRA
|
| 175 |
+
workflow](https://github.com/betweentwomidnights/stable-audio-3/blob/decoder-lora/docs/workflows/decoder-lora.md).
|
| 176 |
Provenance is embedded in the file's `lora_config` metadata. `v1` (identical
|
| 177 |
recipe, `--lambda_patch 0`) is an internal control, not published.
|
| 178 |
|