barelymining commited on
Commit
9dafc5a
·
verified ·
1 Parent(s): 4bfb859

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -1
README.md CHANGED
@@ -14,4 +14,53 @@ tags:
14
  - fasth3
15
  - vsa
16
  - sparse-attention
17
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  - fasth3
15
  - vsa
16
  - sparse-attention
17
+ ---
18
+
19
+ # FastH3 VSA-DataFree LoRA + Gate for ComfyUI (experimental)
20
+
21
+ Two files that let you run **FastVideo's Video Sparse Attention (VSA) 4-step distillation LoRA** for MiniMax H3 inside ComfyUI's native H3 stack, without needing FastVideo's own inference pipeline.
22
+
23
+ > ⚠️ **Experimental.** Tested on one machine (RTX 3090 Ti). Requires the companion custom node — this is not a drop-in for standard LoRA loading. See the node repo before downloading.
24
+
25
+ ## Files
26
+
27
+ - **`fasth3_vsa_v5.safetensors`** (2.05 GB) — the LoRA (backbone + AdaLN projected into the pruned base's compressed AdaLN space, with q/k/v fused into `qkv_proj` to match Comfy's H3 layout)
28
+ - **`fasth3_vsa_gate.safetensors`** (3.6 GB) — 50 `to_gate_compress` weight tensors extracted from FastVideo's VSA-DataFree checkpoint, loaded at runtime by the companion custom node
29
+
30
+ ## Why two files?
31
+
32
+ FastVideo's VSA training added a `to_gate_compress` layer to each transformer block. That layer doesn't exist in ComfyUI's stock H3 model. A LoRA can only modify existing layers — it can't add new ones. So the gate weights ship separately and are injected at runtime by the custom node.
33
+
34
+ ## Required companion custom node
35
+
36
+ You need [ComfyUI-MiniMax-H3-FastVideo](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) — a ComfyUI custom node that injects the gate layers and routes attention through FastVideo's Triton VSA kernel.
37
+
38
+ Installation, requirements, workflow, performance numbers, and known limitations are all documented there.
39
+
40
+ ## Base model
41
+
42
+ Designed for [`minimax_h3_fl2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3) from Comfy-Org. Other pruned INT8 FL2VA variants should also work if their `adaln_t_table` matches. Not tested with the non-pruned or non-INT8 variants.
43
+
44
+ ## Quick usage
45
+
46
+ 1. Install the [companion node](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) and its `vsa` Python dependency
47
+ 2. Place both `.safetensors` files here in `ComfyUI/models/loras/`
48
+ 3. Load with a standard `LoraLoader` (for the `_v5` file) + the `MiniMax H3 VSA (FastVideo + gate)` node (for the gate file)
49
+ 4. 4 steps, CFG 1.0, `topk_ratio` 0.10 for drafts or 1.0 for finals
50
+
51
+ ## Provenance / conversion
52
+
53
+ Converted from FastVideo's [VSA-DataFree checkpoint](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree). Conversion involved:
54
+ - Renaming Diffusers-style keys to Comfy's H3 naming
55
+ - Fusing separate q/k/v LoRA branches into Comfy's `qkv_proj` (block-diagonal B, concatenated A, alpha = 3 × rank)
56
+ - Projecting `adaln_proj.linear.lora_A` from full BF16 space (`[96768, 2688]`) into the pruned model's 8-dim compressed space via least-squares fit against the base's `adaln_t_table` (residual ≈ 0 — mathematically lossless)
57
+ - Extracting `to_gate_compress.set_weight` tensors into a separate file for runtime injection
58
+
59
+ Conversion script (`convert_fastvideo_vsa_to_comfy.py`) is in the [node repo](https://github.com/barelymining/ComfyUI-MiniMax-H3-FastVideo) if you want to re-run it yourself.
60
+
61
+ ## Attribution
62
+
63
+ - Original weights: [FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree)
64
+ - Base model: [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), pruned INT8 by [Comfy-Org](https://huggingface.co/Comfy-Org/MiniMax-H3)
65
+ - VSA kernel: [hao-ai-lab/FastVideo](https://github.com/hao-ai-lab/FastVideo)
66
+ - Distributed under the [MiniMax H3 Community License](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA/blob/main/LICENSE)