Instructions to use MATLOWAI/MiniMax-H3-Motion-Adapter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MATLOWAI/MiniMax-H3-Motion-Adapter with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("MATLOWAI/MiniMax-H3-Motion-Adapter") prompt = "A man with short gray hair plays a red electric guitar." input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
card: state sample sizes (Ref2VA two clips, anime one clip), add the 0.75 fewer-objects trade, add the rooftop smoke test with its caveat
Browse files
README.md
CHANGED
|
@@ -49,8 +49,15 @@ same graph without the adapter and against a same-seed noise floor:
|
|
| 49 |
to 1.23 on all four
|
| 50 |
- blind operator picks, adapter vs no adapter, same seed: 7 for, 0
|
| 51 |
against, 1 no preference
|
| 52 |
-
- it transfers to the Ref2VA checkpoint (
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
Where it is not great (also measured):
|
| 56 |
|
|
@@ -64,14 +71,16 @@ Where it is not great (also measured):
|
|
| 64 |
on full-clip passes; a windowed de-rope on the same creature kept its
|
| 65 |
colour. If the subject is the colour, lower the strength or accept the
|
| 66 |
trade
|
| 67 |
-
- it is anime-heavy in effect: on very fast anime
|
| 68 |
-
every setting; on live-action character
|
| 69 |
-
values below
|
| 70 |
|
| 71 |
## Settings that work
|
| 72 |
|
| 73 |
-
- Strength 1.0. Strength and inject turned out to be the same dial
|
| 74 |
-
|
|
|
|
|
|
|
| 75 |
in-distribution tuning on one fight plate and does not hold elsewhere.
|
| 76 |
- Inject is the dial. Starting points: 0.45 for character or dialogue
|
| 77 |
where the base model over-produces hard; 0.30 where identity or props
|
|
|
|
| 49 |
to 1.23 on all four
|
| 50 |
- blind operator picks, adapter vs no adapter, same seed: 7 for, 0
|
| 51 |
against, 1 no preference
|
| 52 |
+
- it transfers to the Ref2VA checkpoint (span, alternation and jitter
|
| 53 |
+
wins of the same size on the two clips we tried), so one file serves
|
| 54 |
+
both fl2va and ref2va graphs
|
| 55 |
+
- one end-to-end run of the shipped graph on a scene it had never seen
|
| 56 |
+
(the fight choreography moved to a neon rooftop, same seed for both
|
| 57 |
+
arms): alternation 0.24 to 0.19, jitter 0.09 to 0.04, fewer invented
|
| 58 |
+
objects, and it kept the plate's setting where the no-adapter pass
|
| 59 |
+
rewrote the signage. One clip, and each graph at its own shipped inject
|
| 60 |
+
(0.70 vs 0.45), so read it as a smoke test, not a study
|
| 61 |
|
| 62 |
Where it is not great (also measured):
|
| 63 |
|
|
|
|
| 71 |
on full-clip passes; a windowed de-rope on the same creature kept its
|
| 72 |
colour. If the subject is the colour, lower the strength or accept the
|
| 73 |
trade
|
| 74 |
+
- it is anime-heavy in effect: on the one very fast anime clip we
|
| 75 |
+
measured it wins clearly at every setting; on live-action character
|
| 76 |
+
work it needs the lower inject values below
|
| 77 |
|
| 78 |
## Settings that work
|
| 79 |
|
| 80 |
+
- Strength 1.0. Strength and inject turned out to be the same dial. 1.0
|
| 81 |
+
wins on alternation in every paired cell we ran (8 of 8), while 0.75
|
| 82 |
+
invents fewer objects (7 of 8), so 1.0 is the smoothness setting and
|
| 83 |
+
0.75 the lower-invention one; the old 0.75 to 0.80 band was
|
| 84 |
in-distribution tuning on one fight plate and does not hold elsewhere.
|
| 85 |
- Inject is the dial. Starting points: 0.45 for character or dialogue
|
| 86 |
where the base model over-produces hard; 0.30 where identity or props
|