15% More Accurate Than Unsloth: Qwen-3.8-27B Quants for Local Agents
Today we're releasing Qwen3.8-27B-BaseQuant-V1: quantised builds of Qwen3.8-27B in three sizes, each the same size as the matching Unsloth dynamic quant and each closer to the full-precision model. On prompts from six public datasets (coding agents, function calling and chat), BaseQuant has 15% lower KL divergence than Unsloth at both 3 and 4 bits, including UD-Q4_K_M, the size most people run.
| Size | BaseQuant | Same size as |
|---|---|---|
| Small | Qwen3.8-27B-BaseQuant-V1-Small.gguf (10.93 GB) |
Unsloth UD-IQ3_XXS (10.93 GB) |
| Medium | Qwen3.8-27B-BaseQuant-V1-Medium.gguf (16.41 GB) |
Unsloth UD-Q4_K_M (16.46 GB) |
| Large | Qwen3.8-27B-BaseQuant-V1-Large.gguf (20.82 GB) |
Unsloth UD-Q5_K_XL (20.88 GB) |
Why agents need a closer quant
An agent calls the model again and again: plan, call a tool, read the result, call the next one. Every step is conditioned on the steps before it, so a quant that drifts a little from the original at each step drifts a lot by the end of a task. A benchmark score hides this: two quants with the same accuracy can still disagree with the original on many individual answers.
So we measure what an agent actually depends on: how far the quantised model's next-token distribution is from the full-precision model's, at every position, as KL divergence. Lower means the quant behaves more like the model you meant to run.
Results
How much closer, at every size
Public prompts
Each number below is how much lower BaseQuant's KL divergence is than the Unsloth quant of the same size, so bigger is better.
| Public prompts | BQ-Small vs UD-IQ3_XXS | BQ-Medium vs UD-Q4_K_M | BQ-Large vs UD-Q5_K_XL |
|---|---|---|---|
| All six sources | 15% | 15% | 3% |
| OpenHands | 33% | 14% | 1% |
| SWE-agent | 27% | 22% | 18% |
| Glaive function calling | 7% | 13% | 3% |
| OpenOrca | 16% | 18% | 7% |
| Tulu 3 | 14% | 20% | 3% |
| OpenHermes | 6% | 4% | -3% |
The prompts come from two coding-agent trajectory sets, one function-calling set and three chat sets. The full-precision model writes each continuation with thinking on, and both quants are scored along the same text. On the agent and function-calling sources BaseQuant is closer at every size. At the Large size both quants are already very close to the original.
Held-out text
On text that was never used to build the models, the gap is wider:
| Held-out text | BQ-Small vs UD-IQ3_XXS | BQ-Medium vs UD-Q4_K_M | BQ-Large vs UD-Q5_K_XL |
|---|---|---|---|
| Model's own continuations | 21% | 22% | 28% |
| Written text | 18% | 24% | 34% |
| Tool-call turns | 17% | 22% | 29% |
| Tool calls + final answers | 17% | 20% | 26% |
Medium in detail
The 4-bit Medium is the size most people run. Mean KL divergence from the full-precision model (lower is better):
| Workload | Unsloth UD-Q4_K_M | BQ-Medium | Change |
|---|---|---|---|
| All six public sources | 0.00861 | 0.00731 | -15% |
| OpenHands | 0.0212 | 0.0182 | -14% |
| SWE-agent | 0.00912 | 0.00708 | -22% |
| Glaive function calling | 0.0112 | 0.0098 | -13% |
| Held-out: model's own continuations | 0.00713 | 0.00555 | -22% |
| Held-out: written text | 0.2081 | 0.1579 | -24% |
| Held-out: tool-call turns | 0.0127 | 0.0099 | -22% |
| Held-out: tool calls + final answers | 0.0112 | 0.0090 | -20% |
The full per-size numbers are on the model card.
Try it
With llama.cpp:
llama-server -hf basecompute/Qwen3.8-27B-BaseQuant-V1 -hff Qwen3.8-27B-BaseQuant-V1-Medium.gguf -c 32768 -ngl 99
With superfluid, which serves the OpenAI, Anthropic and Ollama APIs on top of it:
superfluid serve basecompute/Qwen3.8-27B-BaseQuant-V1 --runtime llamacpp --pull-file Qwen3.8-27B-BaseQuant-V1-Medium.gguf
The Medium and Large sizes also ship as .base bundles for the BaseRT
engine.
How we measured
- Same size. Each BaseQuant file is no larger than the Unsloth quant it is compared with.
- Same engine. Both quants run in the same llama.cpp build against the same full-precision references.
- Per-token KL divergence from the full-precision model, averaged over every scored position. Intervals are bootstrapped over prompts.
- Held-out text was never used to build the models. The public prompt set is at basecompute/quant-fidelity-eval, with the selection script.
We're also releasing a 4-bit quant of Gemma 4 E4B-it, gemma-4-E4B-it-BaseQuant-V1, with results in the accompanying paper.
