lpalbou commited on
Commit
09e82f8
·
verified ·
1 Parent(s): a6ab1cb

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -23,10 +23,11 @@ for local Apple Silicon inference with
23
  [`mlx-gen`](https://github.com/lpalbou/mlx-gen).
24
 
25
  The source repository ships FP32 weights (~27 GiB) that inference runtimes cast to BF16 at load
26
- time. This repack stores the runtime dtypes directly, halving the download to ~13.7 GiB with no
27
  change in output:
28
 
29
- - UMT5 text encoder: BF16.
 
30
  - Renderer transformer: BF16, with the runtime FP32 keep-set (norms, scale-shift tables, and
31
  embedding layers) stored in FP32 exactly as the loader produces them.
32
  - VAE: FP32, unchanged.
@@ -63,5 +64,5 @@ for workflows, guidance defaults, and task-specific recipes.
63
 
64
  Tensors are bit-exact casts of the pinned source revision
65
  (`ff4c5d4d2d31365c2ffeb30e9753065ee18f58ce`): every tensor equals the value the mlx-gen loader
66
- produces from the FP32 original, so generation output is identical to running from the source
67
- repository at the same settings and seed.
 
23
  [`mlx-gen`](https://github.com/lpalbou/mlx-gen).
24
 
25
  The source repository ships FP32 weights (~27 GiB) that inference runtimes cast to BF16 at load
26
+ time. This repack stores the runtime dtypes directly, reducing the download to ~15.6 GiB (42% smaller) with no
27
  change in output:
28
 
29
+ - UMT5 text encoder: BF16, with the `wo` feed-forward projections stored in FP32 exactly as
30
+ the runtime keeps them (`_keep_in_fp32_modules`).
31
  - Renderer transformer: BF16, with the runtime FP32 keep-set (norms, scale-shift tables, and
32
  embedding layers) stored in FP32 exactly as the loader produces them.
33
  - VAE: FP32, unchanged.
 
64
 
65
  Tensors are bit-exact casts of the pinned source revision
66
  (`ff4c5d4d2d31365c2ffeb30e9753065ee18f58ce`): every tensor equals the value the mlx-gen loader
67
+ produces from the FP32 original, and generation output was verified bit-identical to the source
68
+ repository on image and video use cases at the same settings and seed (max pixel diff 0).