Text-to-Audio
Diffusers
Safetensors
PyTorch
minimax_music3
music-generation
text-to-music
sglang-omni
Instructions to use MiniMaxAI/MiniMax-Music3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MiniMaxAI/MiniMax-Music3 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-Music3", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Add diffusers usage section to the README
Browse filesUsage snippet + low-VRAM offloading for the diffusers weights PR (#2); install pin until huggingface/diffusers#14456 is merged.
README.md
CHANGED
|
@@ -143,6 +143,70 @@ The following end-to-end example contains the complete lyrics, music description
|
|
| 143 |
|---|---|---|
|
| 144 |
| Text-to-music | [View script](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/scripts/end_to_end/minimax_ttm_test.py) | [minimax_ttm.wav](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/assets/minimax_ttm.wav) |
|
| 145 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 146 |
## Prompt Enhancement
|
| 147 |
|
| 148 |
A concise natural-language description can be used directly. For richer prompts and more precise control, use the provided [`music-caption-rewriter`](skills/music-caption-rewriter/SKILL.md) skill to expand it into a Structured Caption containing `Global Metadata`, `Vocal Details`, and `Arrangement`. The skill preserves musical instructions attached to lyric section tags in the arrangement description while keeping the lyric text in the lyrics input.
|
|
|
|
| 143 |
|---|---|---|
|
| 144 |
| Text-to-music | [View script](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/scripts/end_to_end/minimax_ttm_test.py) | [minimax_ttm.wav](https://huggingface.co/MiniMaxAI/MiniMax-Music3/blob/main/assets/minimax_ttm.wav) |
|
| 145 |
|
| 146 |
+
## 🧨 Diffusers
|
| 147 |
+
|
| 148 |
+
MiniMax Music 3 is available as a [diffusers](https://github.com/huggingface/diffusers) modular pipeline. Until [huggingface/diffusers#14456](https://github.com/huggingface/diffusers/pull/14456) is merged, install diffusers from the PR commit:
|
| 149 |
+
|
| 150 |
+
The snippet below fits 24GB+ VRAM GPUs
|
| 151 |
+
|
| 152 |
+
```bash
|
| 153 |
+
pip install git+https://github.com/huggingface/diffusers@dafe3733fcfdbf3c48915fe77be3aef65b5d6a2d transformers accelerate soundfile
|
| 154 |
+
```
|
| 155 |
+
|
| 156 |
+
```python
|
| 157 |
+
import soundfile as sf
|
| 158 |
+
import torch
|
| 159 |
+
from diffusers import ModularPipeline
|
| 160 |
+
|
| 161 |
+
pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-Music3")
|
| 162 |
+
pipe.load_components(dtype=torch.bfloat16)
|
| 163 |
+
pipe.to("cuda")
|
| 164 |
+
|
| 165 |
+
lyrics = """[verse]
|
| 166 |
+
Morning light filtering through the pine
|
| 167 |
+
Every quiet street is yours and mine
|
| 168 |
+
[chorus]
|
| 169 |
+
Softly the world begins to breathe"""
|
| 170 |
+
|
| 171 |
+
prompt = (
|
| 172 |
+
"Genre: acoustic pop. BPM: 96. Key: C major. Warm and intimate, building gently into the chorus. "
|
| 173 |
+
"Vocals: soft female lead, close and breathy, light stacked harmonies in the chorus. "
|
| 174 |
+
"Arrangement: fingerpicked guitar and soft piano; brushed drums and upright bass enter in the chorus."
|
| 175 |
+
)
|
| 176 |
+
|
| 177 |
+
audio = pipe(
|
| 178 |
+
prompt=prompt,
|
| 179 |
+
lyrics=lyrics,
|
| 180 |
+
audio_duration=60.0,
|
| 181 |
+
generator=torch.Generator("cuda").manual_seed(7),
|
| 182 |
+
output="audios",
|
| 183 |
+
)[0]
|
| 184 |
+
|
| 185 |
+
sf.write("song.wav", audio.T.float().cpu().numpy(), pipe.sampling_rate)
|
| 186 |
+
```
|
| 187 |
+
|
| 188 |
+
### Low VRAM
|
| 189 |
+
|
| 190 |
+
The full precision fits under 24GB of VRAM. With automatic CPU offloading, generation takes in ~22 GB; additionally streaming the language model layer by layer makes it fit even 8 GB video cards:
|
| 191 |
+
|
| 192 |
+
```python
|
| 193 |
+
import torch
|
| 194 |
+
from diffusers import ComponentsManager, ModularPipeline
|
| 195 |
+
from diffusers.hooks import apply_group_offloading
|
| 196 |
+
|
| 197 |
+
manager = ComponentsManager()
|
| 198 |
+
manager.enable_auto_cpu_offload(device="cuda")
|
| 199 |
+
pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-Music3", components_manager=manager)
|
| 200 |
+
pipe.load_components(dtype=torch.bfloat16)
|
| 201 |
+
|
| 202 |
+
# Only needed below ~22 GB of VRAM — slower, but fits in 8 GB.
|
| 203 |
+
apply_group_offloading(
|
| 204 |
+
pipe.language_model, onload_device=torch.device("cuda"), offload_type="leaf_level", use_stream=True
|
| 205 |
+
)
|
| 206 |
+
|
| 207 |
+
|
| 208 |
+
```
|
| 209 |
+
|
| 210 |
## Prompt Enhancement
|
| 211 |
|
| 212 |
A concise natural-language description can be used directly. For richer prompts and more precise control, use the provided [`music-caption-rewriter`](skills/music-caption-rewriter/SKILL.md) skill to expand it into a Structured Caption containing `Global Metadata`, `Vocal Details`, and `Arrangement`. The skill preserves musical instructions attached to lyric section tags in the arrangement description while keeping the lyric text in the lyrics input.
|