Instructions to use barelymining/ComfyUI-MiniMax-H3-FastVideo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use barelymining/ComfyUI-MiniMax-H3-FastVideo with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3,FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("barelymining/ComfyUI-MiniMax-H3-FastVideo") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,4 +14,53 @@ tags:
|
|
| 14 |
- fasth3
|
| 15 |
- vsa
|
| 16 |
- sparse-attention
|
| 17 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- fasth3
|
| 15 |
- vsa
|
| 16 |
- sparse-attention
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# FastH3 VSA-DataFree LoRA + Gate for ComfyUI (experimental)
|
| 20 |
+
|
| 21 |
+
Two files that let you run **FastVideo's Video Sparse Attention (VSA) 4-step distillation LoRA** for MiniMax H3 inside ComfyUI's native H3 stack, without needing FastVideo's own inference pipeline.
|
| 22 |
+
|
| 23 |
+
> ⚠️ **Experimental.** Tested on one machine (RTX 3090 Ti). Requires the companion custom node — this is not a drop-in for standard LoRA loading. See the node repo before downloading.
|
| 24 |
+
|
| 25 |
+
## Files
|
| 26 |
+
|
| 27 |
+
- **`fasth3_vsa_v5.safetensors`** (2.05 GB) — the LoRA (backbone + AdaLN projected into the pruned base's compressed AdaLN space, with q/k/v fused into `qkv_proj` to match Comfy's H3 layout)
|
| 28 |
+
- **`fasth3_vsa_gate.safetensors`** (3.6 GB) — 50 `to_gate_compress` weight tensors extracted from FastVideo's VSA-DataFree checkpoint, loaded at runtime by the companion custom node
|
| 29 |
+
|
| 30 |
+
## Why two files?
|
| 31 |
+
|
| 32 |
+
FastVideo's VSA training added a `to_gate_compress` layer to each transformer block. That layer doesn't exist in ComfyUI's stock H3 model. A LoRA can only modify existing layers — it can't add new ones. So the gate weights ship separately and are injected at runtime by the custom node.
|
| 33 |
+
|
| 34 |
+
## Required companion custom node
|
| 35 |
+
|
| 36 |
+
You need [ComfyUI-MiniMax-H3-FastVideo](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) — a ComfyUI custom node that injects the gate layers and routes attention through FastVideo's Triton VSA kernel.
|
| 37 |
+
|
| 38 |
+
Installation, requirements, workflow, performance numbers, and known limitations are all documented there.
|
| 39 |
+
|
| 40 |
+
## Base model
|
| 41 |
+
|
| 42 |
+
Designed for [`minimax_h3_fl2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3) from Comfy-Org. Other pruned INT8 FL2VA variants should also work if their `adaln_t_table` matches. Not tested with the non-pruned or non-INT8 variants.
|
| 43 |
+
|
| 44 |
+
## Quick usage
|
| 45 |
+
|
| 46 |
+
1. Install the [companion node](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) and its `vsa` Python dependency
|
| 47 |
+
2. Place both `.safetensors` files here in `ComfyUI/models/loras/`
|
| 48 |
+
3. Load with a standard `LoraLoader` (for the `_v5` file) + the `MiniMax H3 VSA (FastVideo + gate)` node (for the gate file)
|
| 49 |
+
4. 4 steps, CFG 1.0, `topk_ratio` 0.10 for drafts or 1.0 for finals
|
| 50 |
+
|
| 51 |
+
## Provenance / conversion
|
| 52 |
+
|
| 53 |
+
Converted from FastVideo's [VSA-DataFree checkpoint](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree). Conversion involved:
|
| 54 |
+
- Renaming Diffusers-style keys to Comfy's H3 naming
|
| 55 |
+
- Fusing separate q/k/v LoRA branches into Comfy's `qkv_proj` (block-diagonal B, concatenated A, alpha = 3 × rank)
|
| 56 |
+
- Projecting `adaln_proj.linear.lora_A` from full BF16 space (`[96768, 2688]`) into the pruned model's 8-dim compressed space via least-squares fit against the base's `adaln_t_table` (residual ≈ 0 — mathematically lossless)
|
| 57 |
+
- Extracting `to_gate_compress.set_weight` tensors into a separate file for runtime injection
|
| 58 |
+
|
| 59 |
+
Conversion script (`convert_fastvideo_vsa_to_comfy.py`) is in the [node repo](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) if you want to re-run it yourself.
|
| 60 |
+
|
| 61 |
+
## Attribution
|
| 62 |
+
|
| 63 |
+
- Original weights: [FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree)
|
| 64 |
+
- Base model: [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), pruned INT8 by [Comfy-Org](https://huggingface.co/Comfy-Org/MiniMax-H3)
|
| 65 |
+
- VSA kernel: [hao-ai-lab/FastVideo](https://github.com/hao-ai-lab/FastVideo)
|
| 66 |
+
- Distributed under the [MiniMax H3 Community License](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA/blob/main/LICENSE)
|