Abstract
A decoupled framework for few-step image generation that improves efficiency and performance by separating pixel-space operations from latent denoising training.
Few-step image generation has seen rapid progress, with consistency and meanflow-based methods significantly reducing the number of sampling steps. Despite their low inference cost, these approaches often suffer from training instability and limited scalability. Sphere Encoder is a recent alternative that produces high-quality images in only a few steps; however, it requires repeated transitions between the pixel space and latent space during inference while jointly optimizing reconstruction and generation within a single architecture. This design leads to computational inefficiency and objective conflict between reconstruction and generation. To address these limitations, we decouple the framework into a fixed pretrained image encoder and a separate latent denoising model trained entirely in a spherical latent space. Our approach eliminates repeated pixel-space operations during training and inference, improving efficiency and allowing reconstruction and generation to specialize independently. On Animal-Faces, Oxford-Flowers and ImageNet-1K datasets, our method significantly outperforms Sphere Encoder in both generation quality and inference speed, while achieving competitive results against strong few-step and multi-step baselines.
Community
🚀 Sphere Latent Encoder: Efficient Image Synthesis with Spherical Latent Denoising
This paper proposes Sphere Latent Encoder, an efficient few-step image generation framework that performs denoising entirely in a spherical latent space. Instead of repeatedly moving between pixel space and latent space as in the original Sphere Encoder, the method uses a fixed pretrained representation autoencoder and trains a separate latent denoising model. This decouples reconstruction from generation and makes sampling much more efficient.
Key idea:
Use a pretrained RAE/DINOv2-based encoder as a strong image tokenizer, project noisy latents onto a hypersphere, and train a transformer denoiser directly in that latent space. During inference, the model refines latents over only a few steps and calls the decoder once at the end.
Why it matters:
The approach keeps the simplicity of Sphere Encoder while removing its main bottleneck: repeated encoder-decoder transitions. This leads to substantially lower computational cost and better sample quality in the low-step regime.
Highlights:
- Generates high-quality 256×256 images in only a few sampling steps.
- Reduces inference cost by avoiding repeated pixel-latent conversions.
- Improves over Sphere Encoder on Animal-Faces, Oxford-Flowers, and ImageNet-1K.
- Achieves strong ImageNet-1K results, improving FID from 4.02 to 2.25 at the same 4-step CFG setting, and to 2.11 with 6 steps.
- Ablations show that spherical projection, consistency loss, noise distribution, and the choice of representation autoencoder are all important for performance.
A particularly interesting takeaway is that strong semantic latent representations plus spherical latent modeling can provide a practical alternative to standard diffusion/flow sampling, especially when low-NFE generation is the priority.
Limitations are also clear: the current experiments focus on class-conditional generation, rely on a strong pretrained representation autoencoder, and still find high-quality one-step generation challenging. Overall, this is a promising direction for efficient latent-space generative modeling.
Get this paper in your agent:
hf papers read 2605.15592 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper