Update README.md for reconstruction viewer
Browse files
README.md
CHANGED
|
@@ -18,16 +18,16 @@ keeps the same FCDM decoder family as
|
|
| 18 |
8-block encoder, an 8-block decoder, and a 384-channel spatial latent at
|
| 19 |
`H/32 x W/32`.
|
| 20 |
|
| 21 |
-
Relative to the original
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
alignment instead of the original DINO semantic alignment setup.
|
| 27 |
|
| 28 |
-
|
| 29 |
-
[
|
| 30 |
-
|
|
|
|
| 31 |
|
| 32 |
The p32 checkpoint was trained at `384` resolution rather than the original
|
| 33 |
`256`-scale recipe. With patch size `32`, this gives a `12x12` latent grid
|
|
@@ -43,6 +43,13 @@ covers. Reconstruction uses the default 1-step VP/DDIM path in `bfloat16`.
|
|
| 43 |
|---|---:|---:|---:|---:|---:|
|
| 44 |
| semdisdiffae_p32_v2 | `36.06` | `5.47` | `35.80` | `27.63` | `45.02` |
|
| 45 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
## Throughput
|
| 47 |
|
| 48 |
Measured on an `NVIDIA GeForce RTX 5090` in `bfloat16`, with `5` warmup batches
|
|
@@ -56,12 +63,6 @@ disabled.
|
|
| 56 |
| Encode | `512x512` | `32` | `12.08` | `2649.9` | `579 MiB` |
|
| 57 |
| Decode | `512x512` | `32` | `100.36` | `318.8` | `1042 MiB` |
|
| 58 |
|
| 59 |
-
## Results Viewer
|
| 60 |
-
|
| 61 |
-
The 39-image visual viewer shows originals, FLUX.2 reconstructions,
|
| 62 |
-
semdisdiffae_p32_v2 reconstructions, deltas, and latent PCA side by side:
|
| 63 |
-
[semdisdiffae_p32_v2 results](https://huggingface.co/spaces/data-archetype/semdisdiffae_p32_v2-results).
|
| 64 |
-
|
| 65 |
## Latent Interface
|
| 66 |
|
| 67 |
- `encode()` returns whitened latents using the model's saved running statistics.
|
|
|
|
| 18 |
8-block encoder, an 8-block decoder, and a 384-channel spatial latent at
|
| 19 |
`H/32 x W/32`.
|
| 20 |
|
| 21 |
+
Relative to the original SemDisDiffAE, this model is optimized for a
|
| 22 |
+
lower-resolution latent grid and downstream latent diffusion: patch size `32`
|
| 23 |
+
instead of `16`, `384` latent channels instead of `128`, an 8-block encoder
|
| 24 |
+
instead of a 4-block encoder, and DINOv3 ConvNeXt-B semantic alignment instead
|
| 25 |
+
of the original DINO semantic alignment setup.
|
|
|
|
| 26 |
|
| 27 |
+
For details, see the
|
| 28 |
+
[semdisdiffae_p32_v2 technical report](https://huggingface.co/data-archetype/semdisdiffae_p32_v2/blob/main/technical_report_fcdm_diffae.md).
|
| 29 |
+
For additional shared FCDM / VP decoder background, see the original
|
| 30 |
+
[SemDisDiffAE technical report](https://huggingface.co/data-archetype/semdisdiffae/blob/main/technical_report_semantic.md).
|
| 31 |
|
| 32 |
The p32 checkpoint was trained at `384` resolution rather than the original
|
| 33 |
`256`-scale recipe. With patch size `32`, this gives a `12x12` latent grid
|
|
|
|
| 43 |
|---|---:|---:|---:|---:|---:|
|
| 44 |
| semdisdiffae_p32_v2 | `36.06` | `5.47` | `35.80` | `27.63` | `45.02` |
|
| 45 |
|
| 46 |
+
## Reconstruction Viewer
|
| 47 |
+
|
| 48 |
+
The 39-image reconstruction viewer shows originals, semdisdiffae_p32_v2
|
| 49 |
+
reconstructions, RGB error deltas, and latent PCA side by side, with FLUX.2 VAE
|
| 50 |
+
included for comparison:
|
| 51 |
+
[semdisdiffae_p32_v2 reconstruction viewer](https://huggingface.co/spaces/data-archetype/semdisdiffae_p32_v2-results).
|
| 52 |
+
|
| 53 |
## Throughput
|
| 54 |
|
| 55 |
Measured on an `NVIDIA GeForce RTX 5090` in `bfloat16`, with `5` warmup batches
|
|
|
|
| 63 |
| Encode | `512x512` | `32` | `12.08` | `2649.9` | `579 MiB` |
|
| 64 |
| Decode | `512x512` | `32` | `100.36` | `318.8` | `1042 MiB` |
|
| 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
## Latent Interface
|
| 67 |
|
| 68 |
- `encode()` returns whitened latents using the model's saved running statistics.
|