FLUX.2-klein-9B-MXFP4-RTN-AutoRound

Model Details

This is a MXFP4 (4-bit micro-scaling) quantization of black-forest-labs/FLUX.2-klein-9B, a lightweight distilled text-to-image model. Generated by AutoRound with RTN (round-to-nearest, iters=0).

• Base model: black-forest-labs/FLUX.2-klein-9B
• Quantization: MXFP4 (W4A4), group_size=32
• Method: AutoRound RTN
• Model size: ~7 GB (vs ~18 GB bf16)

Quantization Details

• Scheme: MXFP4 (data_type=mx_fp, bits=4, act_bits=4)
• Group size: 32
• Export format: auto_round (vllm-omni compatible)
• Calibration: coco2014, 4 steps, guidance 1.0

Evaluation

Evaluated with vllm-omni diffusion harness (4 steps, guidance 1.0, 1024×1024, seed 42).

Benchmark	BF16 Baseline	MXFP4 Quantized
DrawBench CLIP	32.81	32.77
DrawBench CLIP-IQA	69.64	68.81
DrawBench ImageReward	1.23	1.18
GenEval	0.856	0.845

MXFP4 quantization shows moderate degradation vs the BF16 baseline (GenEval 0.845 vs 0.856, ImageReward 1.18 vs 1.23). MXFP8 is recommended for higher fidelity.

Usage

from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams

omni = Omni(model="INCModel3/FLUX.2-klein-9B-MXFP4-RTN-AutoRound", mode="text-to-image")
params = OmniDiffusionSamplingParams(
    height=1024, width=1024, seed=42,
    guidance_scale=1.0, num_inference_steps=4, num_outputs_per_prompt=1,
)
out = omni.generate("a red bench in a park", sampling_params_list=[params])

License

Please follow the license of the original model black-forest-labs/FLUX.2-klein-9B.

Produced with autoquant-agent — agent-driven quantize + evaluate + self-heal.