decider-2b-coreai-ft — Mapika/decider-2b fine-tuned on 6,362 verified hard decision items

A decider checkpoint: it reads a state (text or JSON) and typed questions (choice, score, yes/no) and returns a probability for every option from the letter logits at the answer slot. It never generates text. Same architecture, tokenizer, readout and config as Mapika/decider-2b v11; only the weights changed.

What changed: three epochs over 6,362 typed-decision items that we authored and verified for the hard end of the task (long policies, multi-hop rules, dates and numbers, traps). On the 111 public hard items of JevBench it answers 77 correctly against 63 for the v11 checkpoint, measured in the same run with the same code (table below). Easy items are unchanged. On the 72 standard items it answers 62 correctly against 64 for v11.

Results

JevBench public 231 items (easy 48 / standard 72 / hard 111), one question per item, single pass, letter readout through decider.infer.Decider.system_one with use_graphs=False, bf16 on an A100, 2026-09-26. Accuracy = argmax of the returned probabilities.

checkpoint	easy 48	standard 72	hard 111
Mapika/decider-2b v11 (baseline, same run)	1.0000	0.8889	0.5676
previous version of this repo (5,819 items, revision 33b2b241, same run)	1.0000	0.8889	0.6667
this model	1.0000	0.8611	0.6937

This is an independent measurement, not an official JevBench result. The public leaderboard scores include sealed items we cannot run, so the numbers above are not comparable to leaderboard rows. Mapika's own card reports the v11 checkpoint on their serving stack; the baseline row here is our re-measurement in eager mode. Per-item results for every row are in eval/. Earlier checkpoints of this repo: 5,819 items at revision 33b2b241, 1,399 items at revision 5517eb3c.

The fourth authoring round added 543 long documents (long policies and multi-hop rules, median state about 11,800 characters) aimed at JevBench's longest items. On the 12 hard items whose state is 12,000 characters or longer, this model answers 6 correctly, the previous version 5 and v11 6.

Two things to know before relying on the probabilities:

• The fine-tune makes the model more confident on hard items, wrong answers included: mean max-probability on the 111 hard items 0.751 → 0.871, 10-bin ECE 0.183 → 0.181 (from eval/). The temperature in decider_config.json is v11's; it was not refitted.
• Training loss reaches about 0.1 by the third epoch on 6,362 items, so the checkpoint has largely memorised its training set. Held-out accuracy on 100 items from the same authoring pipeline: 0.81 (chance 0.30).

Training data and recipe

• Data: 6,362 items written by Qwen3.8-27B (FP8) from family/topic/length specs, then verified by the same model in five chain-of-thought passes with shuffled options; an item is kept when at least four passes agree with the authored answer. Four authoring rounds (1,065 + 334 + 4,420 + 543 items); the fourth round is long documents only. No JevBench item, public or sealed, is in the training data; the JevBench items were used for evaluation only.
• Recipe: decider.train from Mapika/decider at commit 15ab28e, their delta settings: full fine-tune in bf16, cross-entropy on the letter logits, 3 epochs, lr 8e-6, warmup 20, 12,288 tokens per step with accumulation 2, options shuffled per tokenization, no replay of Mapika's own mixture. 1,545 optimizer steps, about 70 minutes on one A100.

Use

pip install "decider-ai @ git+https://github.com/Mapika/decider@15ab28e"

from decider.infer import Decider
dec = Decider("mlboydaisuke/decider-2b-coreai-ft")   # CUDA: shape-bucketed graphs; use_graphs=False for eager
ans = dec.system_one(state, {"d": {"type": "choice", "instructions": "Which plan applies?", "criteria": {"a": "...", "b": "..."}}})
ans["answers"]["d"]["probabilities"]

Any client of Mapika's POST /v1/systemone server works unchanged: point decider.serve at this repo id.

Licence and attribution

Apache-2.0, like the base model. The base checkpoint, the training code and the inference code are Mapika's (Mapika/decider). The training items were written and verified with Qwen3.8-27B (Apache-2.0).