[docs] simplify FastH3 model card
Browse files
README.md
CHANGED
|
@@ -25,84 +25,43 @@ tags:
|
|
| 25 |
|
| 26 |
# FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree
|
| 27 |
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
90% sparsity.
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
|
|
|
| 35 |
|
| 36 |
-
>
|
| 37 |
-
>
|
| 38 |
-
> 64-token tiles. Dense attention is not a supported substitute.
|
| 39 |
|
| 40 |
-
##
|
| 41 |
-
|
| 42 |
-
- Training run: `20260823T132231Z-v10p5-datafree-sp4`
|
| 43 |
-
- W&B: [a1etree2](https://wandb.ai/wlsaidhi/h3-dmd2-vsa/runs/a1etree2)
|
| 44 |
-
- Step: 1300
|
| 45 |
-
- Transformer content SHA-256: `b36987515e4c75fa4c7aaa632a7842c829ea141b235358a54d782b51230497b3`
|
| 46 |
-
- Checkpoint metadata SHA-256: `dcad0fbee2a7c7e75e53435f4fd98fccf3138844883874edf057962ab48fa428`
|
| 47 |
-
- FastVideo execution commit: `48a047c05ff4138f20cfa33351499c6ec5945f5d`
|
| 48 |
-
|
| 49 |
-
`checkpoint_content.json` records every indexed transformer shard and digest.
|
| 50 |
-
`checkpoint_metadata.json` preserves the source training configuration.
|
| 51 |
-
|
| 52 |
-
## Contents
|
| 53 |
-
|
| 54 |
-
This is a self-contained Diffusers modular-pipeline package. The distilled
|
| 55 |
-
bf16 student is under `transformer/`; the text encoder, tokenizer, processor,
|
| 56 |
-
video VAE, audio VAE, and schedulers are unmodified MiniMax-H3 components.
|
| 57 |
-
|
| 58 |
-
The `transformer_ref` component is not bundled because this student targets
|
| 59 |
-
T2AV. Its modular-index entry remains on `MiniMaxAI/MiniMax-H3`.
|
| 60 |
-
|
| 61 |
-
## Exact FastVideo inference contract
|
| 62 |
-
|
| 63 |
-
The student was trained on the explicit ladder `[999, 749, 500, 250]`. These
|
| 64 |
-
are four transformer forwards; do not replace the ladder with an arbitrary
|
| 65 |
-
four-point scheduler grid.
|
| 66 |
|
| 67 |
```bash
|
| 68 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
|
|
|
| 70 |
python examples/inference/basic/basic_fasth3.py \
|
| 71 |
--model-path FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree \
|
| 72 |
-
--prompt
|
| 73 |
-
--
|
| 74 |
-
--
|
| 75 |
-
--vsa-sparsity 0.9 \
|
| 76 |
-
--vsa-tile-size 64 \
|
| 77 |
-
--vsa-kernel sm100a
|
| 78 |
```
|
| 79 |
|
| 80 |
-
The
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
runs the sparse DiT eagerly; regional compile is disabled for this snapshot.
|
| 84 |
-
|
| 85 |
-
## Training summary
|
| 86 |
-
|
| 87 |
-
- Method: data-free DMD2 with carried backward-simulation rollouts.
|
| 88 |
-
- Student grid: `[999, 749, 500, 250]`.
|
| 89 |
-
- Student attention: VSA-H3, 64-token tiles, 90% sparsity.
|
| 90 |
-
- Prompt conditioning: mixed native resolution and duration T2AV prompts; no
|
| 91 |
-
target video latents are consumed by the data-free objective.
|
| 92 |
-
- Global batch size: 64.
|
| 93 |
-
- Student and critic learning rate: `2e-6`.
|
| 94 |
-
- Precision: fp32 training state and bf16 inference export.
|
| 95 |
-
- Training topology: 32 B200 GPUs, SP=4, full-world FSDP sharding.
|
| 96 |
|
| 97 |
-
##
|
| 98 |
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
- T2AV only; the reference-conditioned transformer was not distilled here.
|
| 104 |
-
- Quality remains below base MiniMax-H3 on difficult motion, fine structure,
|
| 105 |
-
and some audio details.
|
| 106 |
-
- Sampling outside the trained ladder or attention geometry is
|
| 107 |
-
off-distribution.
|
| 108 |
-
- Inherits the MiniMax H3 Community License and base-model limitations.
|
|
|
|
| 25 |
|
| 26 |
# FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree
|
| 27 |
|
| 28 |
+
The recommended FastH3 Preview v1 checkpoint from
|
| 29 |
+
[FastVideo](https://github.com/hao-ai-lab/FastVideo). It generates synchronized
|
| 30 |
+
video and audio from text with four transformer forwards. This step-1300 model
|
| 31 |
+
was trained with data-free DMD2 and VSA-H3 at 90% sparsity.
|
| 32 |
|
| 33 |
+
[Blog](https://haoailab.com/blogs/fasth3-preview/) ·
|
| 34 |
+
[Matching LoRA](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA/tree/main/vsa-datafree) ·
|
| 35 |
+
[FastH3 collection](https://huggingface.co/collections/FastVideo/fastvideo-fasth3)
|
| 36 |
|
| 37 |
+
> This checkpoint requires FastVideo's VSA-H3 attention backend. Use the
|
| 38 |
+
> matching LoRA above if you prefer to download only the distilled adapter.
|
|
|
|
| 39 |
|
| 40 |
+
## Run with FastVideo
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
```bash
|
| 43 |
+
git clone https://github.com/hao-ai-lab/FastVideo.git
|
| 44 |
+
cd FastVideo
|
| 45 |
+
uv venv --python 3.12
|
| 46 |
+
source .venv/bin/activate
|
| 47 |
+
UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"
|
| 48 |
+
```
|
| 49 |
|
| 50 |
+
```bash
|
| 51 |
python examples/inference/basic/basic_fasth3.py \
|
| 52 |
--model-path FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree \
|
| 53 |
+
--prompt "your prompt" \
|
| 54 |
+
--no-warmup \
|
| 55 |
+
--repeats 1
|
|
|
|
|
|
|
|
|
|
| 56 |
```
|
| 57 |
|
| 58 |
+
The script uses FastVideo's optimized four-GPU defaults and the trained
|
| 59 |
+
four-forward schedule. On systems without the Blackwell kernels, add
|
| 60 |
+
`--vsa-kernel triton --no-fa4`; adjust `--num-gpus` for your setup.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
|
| 62 |
+
## Scope
|
| 63 |
|
| 64 |
+
This preview supports text-to-audio-video generation. FL2VA and Ref2VA were
|
| 65 |
+
not distilled. Difficult motion, fine detail, and some audio may remain below
|
| 66 |
+
the base MiniMax H3 model. This checkpoint inherits the
|
| 67 |
+
[MiniMax H3 Community License](LICENSE).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|