Add achievement:fieldnotes tag (blog published)

README.md

@@ -17,6 +17,7 @@ tags:
   - sponsor:nvidia
   - achievement:offbrand
   - achievement:welltuned
+  - achievement:fieldnotes
 ---
 
 <!--

model/spec_lora/README.md

@@ -13,8 +13,11 @@ some nvidia card" — into the structured spec JSON used by
 [FitCheck](https://huggingface.co/spaces/build-small-hackathon/FitCheck), the
 honest "what AI can your computer run" advisor. This powers its paste box.
 
-The one rule that matters: **missing information becomes `null`, never a
-guess.** The model is trained and gated specifically against inventing specs.
+The rule it is trained toward: **missing information should become `null`, not a
+guess.** It is tuned to prefer null over inventing, and does so far more than the
+base model, but it is not perfect: on a builder-blind sealed test it still
+invents a value about 18% of the time it should say null (vs 37% for the base
+model). See Evaluation for the honest numbers.
 
 ## Training data: grounded, not synthetic-echo
 
@@ -27,12 +30,14 @@ comparisons, half-remembered specs, several languages). ~39% of examples have
 no GPU to extract — the don't-invent cases. Trained with Unsloth (bf16 LoRA,
 completion-only loss) on a single RTX 5090 laptop.
 
-## Evaluation: human-written text only
+## Evaluation
+
+### Dev set (human-written, builder-iterated, optimistic)
 
 Evaluated on a 45-example **human-written dev set** (never generator output;
 multilingual, consoles, buying-intent traps, pure refusals). The builder
-iterated against this set, so these are **dev numbers** — optimistically
-biased by adaptive iteration, and labelled as such:
+iterated against this set, so these are **dev numbers**, optimistically biased
+by adaptive iteration and labelled as such:
 
 | round | field accuracy | invented-field rate (hallucination) |
 |---|---|---|
@@ -40,10 +45,34 @@ biased by adaptive iteration, and labelled as such:
 | 3 (answer-only loss + explicit rules) | 85.8% | 12.0% |
 | 5 (final) | **91.6%** | **1.2%** |
 
-A **sealed test set** (written by people who never saw the training data,
-evaluated exactly once, builder-blind) is pending; its result will be added
-here unedited when run. Ship gate: beat the base model zero-shot AND keep the
-invented-field rate under 5% — passed on dev.
+### Sealed test (builder-blind, evaluated once), the honest number
+
+A 40-example sealed test, generated by a separate LLM that never saw the
+training data and evaluated exactly once (machine-generated, so labelled as such
+rather than human-written), checked for zero overlap with train and dev:
+
+| model | field accuracy | invented-field rate |
+|---|---|---|
+| base Qwen3-1.7B, zero-shot | 71.5% | 37.1% |
+| this LoRA | **88.0%** | **17.7%** |
+
+The LoRA clearly beats the base model (accuracy +16.5 points, invented rate
+roughly halved), but it does NOT clear the ship gate's under-5% invented-field
+target on builder-blind data: the real hallucination rate is about 18%, far
+above the 1.2% the adaptively-iterated dev set suggested. Reported unedited,
+because catching exactly that optimism is what a sealed test is for.
+
+Caveat: the sealed labels are machine-generated and unaudited, and some of the
+"inventions" are debatable integrated-graphics cases (the model extracts an iGPU
+the generator marked null), so the absolute figure carries some upward bias; a
+human-audited sealed set would tighten it. The direction is unambiguous.
+
+**Ship gate** (beat base zero-shot AND keep invented-field rate under 5%):
+clears the beat-base half, fails the under-5% half on the sealed set. Treat this
+as a strong extractor that nulls far more often than the base model, not a
+near-zero-hallucination one. Reproduce with `scripts/eval_spec_lora.py
+--testfile <sealed> --baseline <base.json>`; signed result artifacts are in the
+project repo under `artifacts/`.
 
 ## Output schema