Opera-NVFP4

NVFP4 (W4A4) quantization of beyoru/Opera, a finetune of Qwen3.6-27B. 55.6 GB → 22.6 GB (2.5× smaller).

The recipe is reproduced field-for-field from unsloth/Qwen3.6-27B-NVFP4, read back out of its quantization_config, so this checkpoint should behave like that one applied to the Opera weights.

Recipe

compressed-tensors, format: mixed-precision, two groups:

group	weights	activations	targets
group_0 — FP8 W8A8	8-bit float, symmetric, channel, static	8-bit float, symmetric, token, dynamic	self_attn.{q,k,v,o}_proj, linear_attn.{in_proj_qkv,in_proj_z,out_proj}, lm_head, layers 56–63 mlp.{gate,up,down}_proj
group_1 — NVFP4 W4A4	4-bit float, group_size=16, symmetric, tensor_group, static	4-bit float, group_size=16, tensor_group, dynamic=local	all other mlp.{gate,up,down}_proj

ignore: the entire vision tower (model.visual.*, 302 modules) stays bf16.

Three deliberate choices carried over from the reference recipe:

1. The last 8 layers (56–63) keep FP8 instead of dropping to FP4 — final layers are the most sensitive to quantization.
2. All attention and GDN projections stay FP8; only the mid-stack MLP goes to W4A4.
3. Vision tower untouched.

Both groups use dynamic activation scales and weight scales derived from the weights themselves, so no calibration dataset was used.

Reproducing

pip install llmcompressor==0.12.0 compressed-tensors==0.17.1
python quantize_opera_nvfp4.py --model beyoru/Opera --output ./Opera-NVFP4

Two things that will bite you:

• oneshot() needs pipeline="datafree" explicitly. The default (independent) dispatches to sequential, which requires a dataloader to trace the model and dies with TypeError: 'NoneType' object is not iterable. The recipe genuinely needs no calibration data, but llmcompressor will not infer that.
• Load with AutoModelForImageTextToText, not AutoModelForCausalLM. Opera is Qwen3_5ForConditionalGeneration and carries a 333-tensor vision tower.

llmcompressor emits Calibration pipeline is set to 'datafree', but it is recommended to use 'sequential' — the run completes, but that warning is the first place to look if output quality disappoints.

⚠️ Serving support is unverified

This checkpoint has not been load-tested. It uses compressed-tensors, which is the same format that SGLang 0.5.12 refused for unsloth/Qwen3.6-27B-NVFP4 (NotImplementedError: No compressed-tensors compatible scheme was found). A newer vLLM or SGLang is likely required. NVFP4 W4A4 also needs FP4 tensor cores — Blackwell (sm_100 / sm_120 / sm_121); it will not run on Hopper.

Note on the base model

beyoru/Opera is missing the MTP head that Qwen/Qwen3.6-27B ships: 1184 tensors vs 1199, and the 15 absent ones are exactly mtp.*. linear_attn (432) and visual (333) are identical. So this checkpoint cannot do NEXTN speculative decoding — the finetune dropped the draft head. Grafting the 15 tensors back from the base model would restore it, at the cost of an MTP head trained against the base's hidden states rather than the finetuned ones.

Architecture

qwen3_5, 64 layers, hidden 5120, 24 attention heads, full_attention_interval=4 — so 48 of 64 layers use GDN linear attention, which makes the KV cache dramatically cheaper than a comparable full-attention model at long context.