Instructions to use thepatch/same-l-decoder-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Stable Audio 3
How to use thepatch/same-l-decoder-lora with Stable Audio 3:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Rebuild Numbers on the base model with descriptive prompts; add the prompt-distribution result
Browse files
README.md
CHANGED
|
@@ -89,38 +89,37 @@ audio = ae.decode(ae.encode(waveform))
|
|
| 89 |
|
| 90 |
## Numbers
|
| 91 |
|
| 92 |
-
Four generations of 2 minutes
|
| 93 |
-
from 12 candidates by tonality runaway. Each latent is decoded twice,
|
| 94 |
-
re-encodes in place, so dN is N round trips of
|
|
|
|
| 95 |
|
| 96 |
-
| depth | tonality p95 | tonal frames |
|
| 97 |
|---|---|---|---|---|
|
| 98 |
-
| d0, fresh generation |
|
| 99 |
-
| d1, one continuation |
|
| 100 |
-
| d2 | 14.
|
| 101 |
-
| d3 | 17.
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
climbs
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
covering a second, quite different LoRA, at 30 s and 120 s, showed the same
|
| 123 |
-
direction at depth. All of these are fp16 at 120 s.
|
| 124 |
|
| 125 |
## Limits
|
| 126 |
|
|
@@ -140,10 +139,10 @@ direction at depth. All of these are fp16 at 120 s.
|
|
| 140 |
wrong.
|
| 141 |
- **The gain scales with re-encode depth**, so it is small for one-shot
|
| 142 |
generation and compounds across chained continuations.
|
| 143 |
-
- **The squeaks are worst on out-of-distribution prompts**
|
| 144 |
-
|
| 145 |
-
|
| 146 |
-
|
| 147 |
- **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
|
| 148 |
listening. Your material may want a different one.
|
| 149 |
- **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
|
|
|
|
| 89 |
|
| 90 |
## Numbers
|
| 91 |
|
| 92 |
+
Four base-model generations of 2 minutes on ordinary descriptive prompts,
|
| 93 |
+
screened from 12 candidates by tonality runaway. Each latent is decoded twice,
|
| 94 |
+
adapter off and on, and each chain re-encodes in place, so dN is N round trips of
|
| 95 |
+
that decoder config. The adapter is the only variable.
|
| 96 |
|
| 97 |
+
| depth | tonality p95 | tonal frames | 16-22 kHz | SI-SDR vs own d0 |
|
| 98 |
|---|---|---|---|---|
|
| 99 |
+
| d0, fresh generation | 9.80 β **9.10** | 87 β **62** | 3.16 β **4.34** | |
|
| 100 |
+
| d1, one continuation | 11.82 β **9.73** | 462 β **64** | 0.95 β **3.76** | 16.9 β **21.0** |
|
| 101 |
+
| d2 | 14.79 β **10.61** | 2111 β **133** | β1.36 β **3.32** | 12.4 β **16.2** |
|
| 102 |
+
| d3 | 17.85 β **11.74** | 4279 β **453** | β3.36 β **2.96** | 10.1 β **13.6** |
|
| 103 |
+
|
| 104 |
+
The adapter is ahead at every depth, including a fresh generation. Stock tonality
|
| 105 |
+
climbs +8.05 dB over three round trips against the adapter's +2.64, and by d3
|
| 106 |
+
stock carries 4279 tonal frames against 453.
|
| 107 |
+
|
| 108 |
+
It is not doing this by sanding the top end. After three round trips the
|
| 109 |
+
adapter's 16-22 kHz sits 0.2 dB from a **fresh** stock render, at unchanged RMS,
|
| 110 |
+
while stock has lost 6.5 dB of it. Stock's 12-16 kHz meanwhile *rises* with
|
| 111 |
+
depth: the invented energy is replacing the air.
|
| 112 |
+
|
| 113 |
+
Prompt distribution matters more here than anything else. The same ladder run on
|
| 114 |
+
terse genre-tag prompts, which are out of distribution for the base model, starts
|
| 115 |
+
at **1649** tonal frames instead of 87. With a DiT LoRA attached, which makes
|
| 116 |
+
those tags in-distribution again, it starts at 780. The artifact is worst where
|
| 117 |
+
the prompt sits furthest from what the model expects, and re-encoding compounds
|
| 118 |
+
whatever it starts with.
|
| 119 |
+
|
| 120 |
+
A DiT LoRA does not change the shape: on that run stock climbs +5.79 dB against
|
| 121 |
+
the adapter's +0.73, and the adapter ends d3 holding more 12-16 kHz and 16-22 kHz
|
| 122 |
+
than a fresh stock render. All runs are fp16 at 120 s.
|
|
|
|
|
|
|
| 123 |
|
| 124 |
## Limits
|
| 125 |
|
|
|
|
| 139 |
wrong.
|
| 140 |
- **The gain scales with re-encode depth**, so it is small for one-shot
|
| 141 |
generation and compounds across chained continuations.
|
| 142 |
+
- **The squeaks are worst on out-of-distribution prompts.** Genre-tag prompts on
|
| 143 |
+
the base model start with 19x more tonal frames than descriptive ones. A DiT
|
| 144 |
+
LoRA makes such tags in-distribution again and lands in between. This adapter
|
| 145 |
+
reduces the artifact; it does not make prompt choice irrelevant.
|
| 146 |
- **One checkpoint, one dataset**, step 2000, chosen on re-encode behaviour and
|
| 147 |
listening. Your material may want a different one.
|
| 148 |
- **Neither half of the autoencoder is bit-deterministic** (encoder 1.5 %
|