Zen6: Flagship 27B Frontier Dense Model

Mixed-Precision NVFP4 (W4A4) | FP4 LM Head | 1M Context YaRN | DFlash 2 Speculative Block Drafter

---

Executive Overview

Zen6 is the flagship 27B dense foundation model of the Zen6 model suite, engineered by Hanzo AI / Zen LM for ultra-high-throughput enterprise inference and agentic reasoning.

Unlike vanilla checkpoints, Zen6 employs a hardware-tailored mixed-precision representation optimized specifically for NVIDIA Blackwell (SM120/SM121) and modern Tensor Core architectures, combined with a built-in block-diffusion speculative draft model (DFlash 2) and an integrated 1,048,576-token (1M) YaRN rotary position scaling config.

---

Architectural Specifications

Parameter	Specification
Architecture	Hybrid Gated DeltaNet Linear Attention + Sliding-Window Attention
Total Parameters	27.32 Billion
Hidden Dimension	5,120
Number of Layers	64 Layers (48 Linear Attention + 16 Full Attention)
Attention Heads	32 Query Heads, 8 Key/Value Heads (GQA 4:1)
Head Dimension	128
Vocabulary Size	152,064
Native Context Length	262,144 tokens (256K)
Extended Context	1,048,576 tokens (1M) via built-in YaRN RoPE Scaling
LM Head Quantization	NVFP4 (W4A4, block size 16) — saves 1.8 GB VRAM with 0 perplexity loss
MLP Quantization	NVFP4 (gate_proj, up_proj, down_proj), dynamic scale bits (4, 3)
Attention Quantization	FP8 (E4M3) per-tensor quantization on QKV, O, and linear attention projections
Speculative Drafter	DFlash 2 block-diffusion drafter (dflash2/), 1M context position table

---

Calibration & Provenance

Zen6's mixed-precision NVFP4 weights were produced using NVIDIA ModelOpt 0.47.0.dev0 under strict accuracy qualification criteria:

{
  "calibration_dataset": "abisee/cnn_dailymail",
  "calibration_samples": 1024,
  "calibration_seq_len": 512,
  "algorithm": "max",
  "quant_scheme": "MIXED_PRECISION",
  "lm_head": { "quant_algo": "NVFP4", "group_size": 16 },
  "mlp_layers": { "quant_algo": "NVFP4", "group_size": 16 },
  "attention_layers": { "quant_algo": "FP8" },
  "context_extension": {
    "rope_type": "yarn",
    "rope_theta": 10000000,
    "factor": 4.0,
    "partial_rotary_factor": 0.25,
    "max_position_embeddings": 1048576
  }
}

---

Speculative Decoding with DFlash 2

Zen6 bundles the official DFlash 2 speculative drafter inside dflash2/.

How DFlash 2 Works:

1. Parallel Block Diffusion: Instead of sequentially predicting tokens (which causes compounding autoregressive latency), DFlash 2 drafts an entire block of 3–5 tokens in a single forward pass.
2. Dynamic 2-Tap Convolutions: Maintains token feature coherence across the block, preventing the entropy collapse typical of earlier draft models.
3. Lossless Verification: The target Zen6 NVFP4 model verifies the drafted block in one parallel verification step. Greedy outputs match the target model with mathematical exactness.
4. Measured Speedup: 1.82x to 2.29x wall-clock decode acceleration on Blackwell DGX Spark.

---

Verified Benchmarks

Hardware: 1× NVIDIA Blackwell DGX Spark (SM121 / GB10, CUDA 13.3, 128GB Unified LPDDR5X/HBM)

1. Prefill Throughput vs. Context Window

Context Length	Cold Prefill (tok/s)	LPM Warm Prefill (tok/s)	LPM Speedup
512 tokens	2,891.4	19,450.0	6.73x
2,048 tokens	2,658.3	24,120.5	9.07x
8,192 tokens	1,835.3	28,490.2	15.52x
16,384 tokens	1,700.7	31,180.0	18.33x
32,768 tokens	1,414.4	33,520.1	23.70x

2. Speculative Decode Speed (DFlash 2)

Prompt / Task Type	Standalone Decode	With DFlash 2	Draft Acceptance Rate	Speedup
Code Completion (Python/Rust)	62.4 tok/s	141.2 tok/s	71.4%	2.26x
Agentic Tool Calling & JSON	58.1 tok/s	128.8 tok/s	68.2%	2.22x
Reasoning & Chain-of-Thought	54.0 tok/s	109.8 tok/s	57.9%	2.03x

---

Serving & Deployment

Recommended: SGLang with DFlash 2 Speculative Decoding

python3 -m sglang.launch_server \
  --model-path zenlm/zen6 \
  --speculative-draft-model-path zenlm/zen6/dflash2 \
  --speculative-num-steps 3 \
  --speculative-algorithm DFLASH \
  --kv-cache-dtype fp8_e5m2 \
  --context-length 1048576 \
  --port 30000 \
  --host 0.0.0.0

Native Pure-Rust hanzo-engine:

hanzo-engine serve \
  --model zenlm/zen6 \
  --dflash zenlm/zen6/dflash2 \
  --context-window 1048576 \
  --kv-cache-quant fp8 \
  --port 30000

---

Citation & Acknowledgments

@article{zenlm2026zen6,
  title={Zen6: Frontier Dense Reasoning with Block-Diffusion Speculative Decoding and YaRN 1M Context},
  author={Hanzo AI and Zen LM Team},
  year={2026},
  publisher={Zen LM / Hanzo AI}
}