Instructions to use MATLOWAI/MiniMax-H3-Motion-Adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MATLOWAI/MiniMax-H3-Motion-Adapter with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("MATLOWAI/MiniMax-H3-Motion-Adapter") prompt = "A man with short gray hair plays a red electric guitar." input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
card: say plainly that the LoRA adds no compute; the nodes stretch the clock, the LoRA changes behaviour on it
Browse files
README.md
CHANGED
|
@@ -32,7 +32,12 @@ known bad habit: on very fast motion the base model over-produces, and the
|
|
| 32 |
result alternates between advance and snap frame to frame. This adapter
|
| 33 |
was trained on that exact task (frames held out of the middle of a burst,
|
| 34 |
the model asked to fill them back in), and it teaches the base model to
|
| 35 |
-
spend the extra clock on smoothness instead of invention.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
Measured on four clips the adapter never saw during training (a fight,
|
| 38 |
an anime sword ring, a character swap, a dialogue close-up), against the
|
|
|
|
| 32 |
result alternates between advance and snap frame to frame. This adapter
|
| 33 |
was trained on that exact task (frames held out of the middle of a burst,
|
| 34 |
the model asked to fill them back in), and it teaches the base model to
|
| 35 |
+
spend the extra clock on smoothness instead of invention. To be precise
|
| 36 |
+
about the mechanics: the LoRA is a plain rank-16 weight delta and adds no
|
| 37 |
+
compute and needs no custom node. The extra clock comes from the nodes
|
| 38 |
+
(the de-rope stretches the burst into more latent tokens); the LoRA
|
| 39 |
+
changes how the model behaves on that stretched clock, because it was
|
| 40 |
+
trained in that exact situation.
|
| 41 |
|
| 42 |
Measured on four clips the adapter never saw during training (a fight,
|
| 43 |
an anime sword ring, a character swap, a dialogue close-up), against the
|