lpalbou commited on
Commit
1f246a7
·
verified ·
1 Parent(s): 3dd7f5d

Update TI2V-5B memory and validation card

Browse files
Files changed (1) hide show
  1. README.md +48 -19
README.md CHANGED
@@ -9,6 +9,7 @@ tags:
9
  - mflux
10
  - apple-silicon
11
  - 8-bit
 
12
  - wan
13
  - wan2.2
14
  - video-generation
@@ -17,37 +18,62 @@ tags:
17
  ---
18
  # wan2.2-ti2v-5b-diffusers-8bit
19
 
20
- This repository contains MLX-Gen saved weights for `Wan-AI/Wan2.2-TI2V-5B-Diffusers`. The checkpoint is designed for local Apple Silicon inference with [`mlx-gen`](https://github.com/lpalbou/mlx-gen).
 
 
 
21
 
22
- It uses the mflux/MLX saved-weight layout and MLX quantization tensors. It is not a Diffusers or Transformers `from_pretrained()` checkpoint.
 
23
 
24
  ## Source Model
25
 
26
  Original model: [`Wan-AI/Wan2.2-TI2V-5B-Diffusers`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers).
27
 
28
- ## License and Access
29
-
30
  This quantized derivative follows the Apache 2.0 license of the source model.
31
 
32
  ## Quantization
33
 
34
- This is an MLX q8 checkpoint for Wan2.2 TI2V-5B. MLX-Gen uses 8-bit quantization for Wan modules where MLX supports quantization:
35
 
36
- - q8 for quantizable Wan transformer attention and feed-forward modules.
37
  - BF16 for the Wan VAE.
38
- - BF16 for Wan transformer conditioning/output projection linears, the UMT5 text encoder, scheduler metadata, tokenizer files, norms, convolutions, and other non-quantizable parameters.
 
 
 
 
 
 
 
 
39
 
40
- Wan q4 quality and any possible mixed q4/q8 policy are still under validation. Prefer q8 for publishable Wan checkpoints until the q4 policy is documented.
41
 
42
- See the [MLX-Gen quantization docs](https://github.com/lpalbou/mlx-gen/blob/main/docs/quantization.md) for compatibility notes.
 
43
 
44
- ## Compatibility
 
 
 
 
45
 
46
- Requires `mlx-gen >= 0.18.6`.
 
 
47
 
48
- Generated with `mlx-gen 0.18.6`.
 
49
 
50
- Use the `mlxgen` command and Python import path for new MLX-Gen projects.
 
 
 
 
 
 
 
51
 
52
  ## Usage
53
 
@@ -58,20 +84,23 @@ mlxgen download --model AbstractFramework/wan2.2-ti2v-5b-diffusers-8bit
58
 
59
  mlxgen generate \
60
  --model AbstractFramework/wan2.2-ti2v-5b-diffusers-8bit \
61
- --task text-to-video \
62
- --prompt "Your video prompt here" \
63
  --width 1280 \
64
  --height 704 \
65
- --frames 121 \
66
- --steps 50 \
67
  --guidance 5 \
68
  --fps 24 \
69
- --seed 42 \
70
  --output video.mp4
71
  ```
72
 
 
 
73
  ## Attribution
74
 
75
- MLX-Gen is based on [mflux](https://github.com/filipstrand/mflux) by Filip Strand and the original mflux contributors. This model card is generated by MLX-Gen so derived checkpoints keep that attribution visible.
 
76
 
77
  Quantized and contributed by [@lpalbou](https://huggingface.co/lpalbou).
 
9
  - mflux
10
  - apple-silicon
11
  - 8-bit
12
+ - mixed-q8-bf16
13
  - wan
14
  - wan2.2
15
  - video-generation
 
18
  ---
19
  # wan2.2-ti2v-5b-diffusers-8bit
20
 
21
+ This repository contains mixed q8/BF16 MLX-Gen saved weights for
22
+ [`Wan-AI/Wan2.2-TI2V-5B-Diffusers`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers).
23
+ It is designed for local Apple Silicon inference with
24
+ [`mlx-gen`](https://github.com/lpalbou/mlx-gen).
25
 
26
+ It uses the mflux/MLX saved-weight layout with MLX quantization tensors. It is not a Diffusers or
27
+ Transformers `from_pretrained()` checkpoint.
28
 
29
  ## Source Model
30
 
31
  Original model: [`Wan-AI/Wan2.2-TI2V-5B-Diffusers`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B-Diffusers).
32
 
 
 
33
  This quantized derivative follows the Apache 2.0 license of the source model.
34
 
35
  ## Quantization
36
 
37
+ This is a mixed q8/BF16 checkpoint:
38
 
39
+ - q8 for quantizable Wan transformer attention and feed-forward linears.
40
  - BF16 for the Wan VAE.
41
+ - BF16 for Wan transformer `condition_embedder.*` and `proj_out`.
42
+ - BF16 for the UMT5 text encoder, scheduler metadata, tokenizer files, norms, convolutions, and
43
+ other non-quantizable parameters.
44
+
45
+ The upstream TI2V-5B source snapshot is not uniformly 16-bit on disk: the transformer and VAE
46
+ safetensors are FP32, while the UMT5 text encoder is BF16. MLX-Gen loads Wan transformer/VAE
47
+ weights at BF16 runtime precision.
48
+
49
+ ## Measurements
50
 
51
+ Measured on 2026-06-04 with `mlx-gen 0.18.10` on an Apple M5 Max with 128 GiB unified memory.
52
 
53
+ Validation profile: `1280x704`, 17 frames, 20 denoising steps, guidance `5`, 24 fps, seed `321`,
54
+ explicit empty negative prompt.
55
 
56
+ | Layout | Storage | Logical Model | Full-Process Physical Peak | Max RSS | MLX Peak | Total Time | Output |
57
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | --- |
58
+ | Upstream source snapshot | 31.9 GiB | 10.6 GiB | 102.7 GiB | 13.7 GiB | 58.5 GiB | 216.2 s | [base-source.mp4](validation/ti2v5b-clean/base-source.mp4) |
59
+ | Prepared BF16 package | 21.2 GiB | 10.6 GiB | 102.6 GiB | 14.5 GiB | 58.5 GiB | 261.6 s | [prepared-bf16.mp4](validation/ti2v5b-clean/prepared-bf16.mp4) |
60
+ | This mixed q8/BF16 package | 16.9 GiB | 6.3 GiB | 103.7 GiB | 13.8 GiB | 54.2 GiB | 243.4 s | [mixed-q8-bf16.mp4](validation/ti2v5b-clean/mixed-q8-bf16.mp4) |
61
 
62
+ This package reduces storage, logical model bytes, active MLX model bytes, and MLX allocator peak in
63
+ the validation profile. It did not reduce full-process physical peak memory in this profile because
64
+ transient video-generation allocations dominated the run.
65
 
66
+ The source and prepared BF16 package produced byte-identical decoded MP4 frames. This mixed q8/BF16
67
+ package stayed visually in the same family with mean frame MAE `1.66` versus source/BF16.
68
 
69
+ `Storage` is the Hugging Face repository total. `Logical Model` is the loaded Wan transformer plus
70
+ VAE tensor footprint measured from MLX arrays. `Full-Process Physical Peak` is Darwin
71
+ `phys_footprint` sampled from model initialization through MP4 save and health validation.
72
+
73
+ Validation assets:
74
+
75
+ - [contact-sheet.png](validation/ti2v5b-clean/contact-sheet.png)
76
+ - [metrics.json](validation/ti2v5b-clean/metrics.json)
77
 
78
  ## Usage
79
 
 
84
 
85
  mlxgen generate \
86
  --model AbstractFramework/wan2.2-ti2v-5b-diffusers-8bit \
87
+ --prompt "A short cinematic video of a glowing orange glass sphere floating above calm teal water, soft reflections, gentle camera movement" \
88
+ --negative-prompt "" \
89
  --width 1280 \
90
  --height 704 \
91
+ --frames 17 \
92
+ --steps 20 \
93
  --guidance 5 \
94
  --fps 24 \
95
+ --seed 321 \
96
  --output video.mp4
97
  ```
98
 
99
+ TI2V-5B also supports first-frame image-to-video in MLX-Gen when one input image is supplied.
100
+
101
  ## Attribution
102
 
103
+ MLX-Gen is based on [mflux](https://github.com/filipstrand/mflux) by Filip Strand and the original
104
+ mflux contributors.
105
 
106
  Quantized and contributed by [@lpalbou](https://huggingface.co/lpalbou).