File size: 4,790 Bytes
0076fee 5186127 0076fee afb77db 0076fee | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | ---
license: other
license_name: minimax-h3-license
license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
base_model: MiniMaxAI/MiniMax-H3
tags:
- svdquant
- w4a4
- int4
- video
- text-to-video
- quantized
---
# MiniMax-H3 · SVDQuant W4A4 (int4, rank 32, GPTQ)
4-bit **weights and activations** for the MiniMax-H3 31B video+audio
transformer — true [SVDQuant](http://arxiv.org/abs/2411.05007) (ICLR 2025
Spotlight): activation outliers absorbed into a 16-bit rank-32 low-rank
branch, the residual GPTQ-rounded to int4, activations quantized to int4
per-token at runtime, executed on fused CUTLASS tensor-core kernels. This
is not weight-only quantization.
The text conditioner (Qwen3-VL 31B) ships W4A16+GPTQ in the same release:
**every stored weight is int4**; only norms, embeddings, and the visual
tower stay bf16, matching MiniMax's own quantization recipe.
## Measured (NVIDIA A100 80GB, 124 frames @ 24fps, 960x544)
| | BF16 | this release | factor |
|---|---|---|---|
| DiT checkpoint | 61.7 GB | **19.6 GB** | 3.15x |
| generation wall-clock | 484 s | **369 s** | **1.31x faster** |
| vs unfused reference dequant | 2427 s | 369 s | 6.6x |
| quantized GEMM (layer level) | — | — | 1.37-1.38x |
**All-resident configuration** (this release's DiT + TE together,
no CPU offload — unreachable for BF16 on one 80GB card):
| | BF16 (offloaded) | all-int4 resident | factor |
|---|---|---|---|
| generation wall-clock | 484 s | **318 s** | **1.52x faster** |
| pipeline VRAM | 65 GB peak, offload churn | 48.9 GB steady, 54.3 peak | fits |
| DiT + TE weights on disk | 123.8 GB | 37.6 GB | 3.3x |
Conversion cost: 44 min for the DiT (18 calib + 26 GPTQ) on one A100.
Kernel outputs agree with the fp32 reference oracle to 1.5-2.1% (the
bf16-vs-fp32 activation-rounding delta) at every layer shape.
Quality: same-seed renders are visually indistinguishable from BF16
(samples in this repo). On the Z-Image anchor, the same pipeline's GPTQ
checkpoint scores **better LPIPS than the officially published nunchaku
checkpoint** (0.288 vs 0.334).
## Before / after (same seed, sound on)
| BF16 original | this release (all-int4) |
|---|---|
| <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p0_bf16.mp4"></video> | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p0_ours_int4.mp4"></video> |
| <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p1_bf16.mp4"></video> | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p1_ours_int4.mp4"></video> |
Prompts: a red fox in a snowy pine forest; a saxophonist under neon in
light rain. Video and soundtrack are generated jointly by the model.
## Use
```bash
pip install git+https://github.com/ModelsLab/svdquant git+https://github.com/rootonchair/nunchaku-lite
```
```python
import svdquant
transformer = svdquant.load_model("minimax-h3-packed.safetensors") # this repo's file
# drop into the diffusers ModularPipeline in place of the BF16 transformer
```
Requires an int4-tensor-core GPU (sm_75-89: RTX 20/30/40, A100) and
torch >= 2.11. An NVFP4 sibling for RTX 50-series (sm_120 has no int4
path) is planned from a fresh BF16 pass — int4 and fp4 grids do not nest,
so transcoding is never used.
## Honest notes
1. Attention stays bf16 — at video sequence lengths it bounds the
end-to-end speedup (Amdahl); the 1.31x reflects that.
2. The `token_refiner` (2 blocks, ~2% of params) stays bf16, following
MiniMax's own int8 recipe.
3. The AWQ repack of the 50 modulation layers re-derives scales; groups
GPTQ pushed to -8 take one extra bounded rounding.
## Independently verified (cold download)
Reproduced on a **separate, fresh A100 80GB** by downloading only this
repo's files plus the base model — the exact user path:
| | BF16 (offloaded) | this release (resident) |
|---|---|---|
| generation | 496.7 s | **350.9 s (1.42x)** |
| peak VRAM | 65.2 GB | 54.6 GB (49.2 steady) |
Raw log: `verified.json` in this repo. First-build A100 measured 1.52x;
host-to-host variance puts the honest claim at **1.4-1.5x**.
## Credits and license
Weights derive from [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3)
and inherit its license. Method: SVDQuant (Li et al., MIT HAN Lab).
Kernels and packed layout: [nunchaku](https://github.com/nunchaku-ai/nunchaku),
[nunchaku-lite](https://github.com/rootonchair/nunchaku-lite) and
[diffuse-compressor](https://github.com/rootonchair/diffuse-compressor)
by rootonchair (Apache-2.0, vendored with attribution). Quantized with
[svdquant](https://github.com/ModelsLab/svdquant) by ModelsLab.
|