feat(hints): ADR-009 layered HintGenerator; accepted

Track B of the deep-work-loop. Extends hint_generator.py from a flat
dispatch registry into a layered, cost-ordered HintGenerator behind a typed
Protocol — the SDPO textual-feedback hint source (Composer 2.5's hint
mechanism is unstated in every Cursor artifact; this is our reconstruction).

Layers (tried cheapest-first, first non-None wins):
1. TemplateHintGenerator — the existing 5 templates (free, deterministic;
byte-identical to dispatch(), preserved).
2. RawErrorHintGenerator — raw env/tool error text as the hint (free;
covers any message-bearing site templates miss). SDPO "environment
feedback as conditioning signal".
3. LLMJudgeHintGenerator — <=2-sentence corrective hint via an injected
complete() callable (optional, OFF unless provided; in-memory + disk
cache keyed on error-context hash; covers style/communication/effort).

CompositeHintGenerator.as_collator_hook() returns a callable matching the
existing CollatorConfig.hint_generator signature (error_kind, error_meta)
-> str|None — ZERO collator change. default_composite() builds the
recommended stack.

Sibling-bootstrap: satisfied-by-design — it needs sibling rollouts (RL path
only), so it's a trainer-side concern, not a HintContext layer.

12 new tests (Protocol, byte-identity, cost-ordering w/ zero-LLM on
template sites, cache, collator-hook drop-in) — all green. All 6 ADR-009
gates green -> accepted.

composer_replication/hint_generator.py

@@ -104,4 +104,214 @@ def register(error_kind: str, fn: Callable[[HintContext], str]) -> None:
     HINT_TEMPLATES[error_kind] = fn
 
 
-__all__ = ["dispatch", "register", "HintContext", "HINT_TEMPLATES"]
+# ===========================================================================
+# Layered HintGenerator architecture (ADR-009)
+# ===========================================================================
+#
+# Composer 2.5 inserts a natural-language hint at each error turn; the
+# hint-conditioned forward becomes the SDPO teacher. HOW Cursor generates the
+# hint is unstated in every Cursor artifact (both blogs + the Composer 2 tech
+# report, arXiv:2603.24477 — confirmed absent in research/10). So this is our
+# design problem. The cited papers bracket the answer: OPSD conditions the
+# teacher on ground-truth; SDPO generalizes to environment feedback and the
+# "successful sibling rollout as implicit feedback" trick.
+#
+# We implement a layered generator, tried cheapest-first:
+#   1. TemplateHintGenerator   — the registry above (free, deterministic;
+#      covers tool-error classes). The first layer.
+#   2. RawErrorHintGenerator   — wrap the raw env/tool error text as the hint
+#      (free; covers any error with a message but unmatched by a template).
+#   3. LLMJudgeHintGenerator   — an LLM produces a <=2-sentence corrective hint
+#      (cost ~$0.0005/site; covers style/communication/effort sites templates
+#      can't). Cached on disk; optional; OFF unless a client is provided.
+#   4. (sibling-bootstrap)     — RL-rollout-path only; not a HintContext-driven
+#      layer (needs sibling rollouts), exposed as a flag for the trainer to use.
+#
+# All layers satisfy the HintGenerator Protocol and compose via
+# CompositeHintGenerator, whose .as_collator_hook() returns a callable matching
+# the collator's existing `hint_generator: Callable[[str, dict], str | None]`
+# hook — ZERO collator change.
+
+from typing import Protocol, runtime_checkable
+
+
+@runtime_checkable
+class HintGenerator(Protocol):
+    """A hint source. Returns hint text for an error context, or None to defer
+    to the next layer."""
+
+    def generate(self, error_kind: str, error_meta: dict) -> str | None: ...
+
+
+class TemplateHintGenerator:
+    """Layer 1: the existing template registry. Free, deterministic.
+
+    Preserves the exact behavior of the module-level `dispatch()` so existing
+    callers and tests see no change.
+    """
+
+    def generate(self, error_kind: str, error_meta: dict) -> str | None:
+        # `dispatch` reads HintContext keys; error_meta IS that context dict
+        # plus the kind. Merge so templates that read `error_kind` still work.
+        ctx: HintContext = dict(error_meta)  # type: ignore[assignment]
+        ctx.setdefault("error_kind", error_kind)
+        return dispatch(error_kind, ctx)
+
+
+class RawErrorHintGenerator:
+    """Layer 2: use the raw env/tool error text itself as the hint.
+
+    Covers any error site that carries a message but isn't matched by a
+    template. Free. SDPO's "environment feedback as the conditioning signal"
+    (arXiv:2601.20802) — the rawest form of that.
+    """
+
+    def __init__(self, max_chars: int = 500) -> None:
+        self.max_chars = max_chars
+
+    def generate(self, error_kind: str, error_meta: dict) -> str | None:
+        msg = error_meta.get("error_message") or error_meta.get("error") or ""
+        msg = str(msg).strip()
+        if not msg:
+            return None
+        truncated = msg[: self.max_chars]
+        return f"Reminder: the previous action produced this error:\n{truncated}\nReconsider and retry."
+
+
+class LLMJudgeHintGenerator:
+    """Layer 3: an LLM produces a short corrective hint.
+
+    Covers style/communication/effort sites that templates can't. Optional and
+    OFF unless a `complete` callable is provided. Results are cached on disk
+    keyed on a hash of the error context (so repeated identical sites cost
+    nothing after the first).
+
+    `complete(prompt: str) -> str` is an injected text-completion callable
+    (e.g. an OpenRouter chat wrapper). Kept abstract so this module has no hard
+    network dependency and is unit-testable with a stub.
+    """
+
+    PROMPT_TEMPLATE = (
+        "An autonomous coding agent made a mistake at one step of a trajectory. "
+        "Write a SHORT (<=2 sentences) corrective hint that, if the agent had "
+        "seen it, would steer it to the right behavior for THIS step only. Do "
+        "not solve the whole task; just correct the local mistake.\n\n"
+        "Error kind: {error_kind}\n"
+        "Error / context:\n{error_message}\n\n"
+        "Corrective hint:"
+    )
+
+    def __init__(
+        self,
+        complete: Callable[[str], str] | None = None,
+        *,
+        cache_dir: str | None = None,
+    ) -> None:
+        self.complete = complete
+        self._cache_dir = cache_dir
+        self._mem_cache: dict[str, str] = {}
+
+    def _cache_key(self, error_kind: str, error_meta: dict) -> str:
+        import hashlib
+        import json
+
+        blob = json.dumps(
+            {"k": error_kind, "m": error_meta}, sort_keys=True, default=str
+        )
+        return hashlib.sha256(blob.encode("utf-8")).hexdigest()[:32]
+
+    def _disk_get(self, key: str) -> str | None:
+        if not self._cache_dir:
+            return None
+        from pathlib import Path
+
+        p = Path(self._cache_dir) / f"{key}.txt"
+        return p.read_text(encoding="utf-8") if p.exists() else None
+
+    def _disk_put(self, key: str, value: str) -> None:
+        if not self._cache_dir:
+            return
+        from pathlib import Path
+
+        d = Path(self._cache_dir)
+        d.mkdir(parents=True, exist_ok=True)
+        (d / f"{key}.txt").write_text(value, encoding="utf-8")
+
+    def generate(self, error_kind: str, error_meta: dict) -> str | None:
+        if self.complete is None:
+            return None  # judge disabled — defer
+        key = self._cache_key(error_kind, error_meta)
+        if key in self._mem_cache:
+            return self._mem_cache[key]
+        cached = self._disk_get(key)
+        if cached is not None:
+            self._mem_cache[key] = cached
+            return cached
+        prompt = self.PROMPT_TEMPLATE.format(
+            error_kind=error_kind,
+            error_message=str(error_meta.get("error_message")
+                              or error_meta.get("error") or "(no message)")[:1000],
+        )
+        hint = self.complete(prompt).strip()
+        if not hint:
+            return None
+        self._mem_cache[key] = hint
+        self._disk_put(key, hint)
+        return hint
+
+
+class CompositeHintGenerator:
+    """Tries each layer in order, returning the first non-None hint.
+
+    Order is cost-ascending: templates (free) -> raw error (free) -> LLM judge
+    (paid, optional). The first layer to produce a hint wins, so the common
+    tool-error case never reaches the LLM.
+    """
+
+    def __init__(self, layers: list[HintGenerator]) -> None:
+        self.layers = layers
+
+    def generate(self, error_kind: str, error_meta: dict) -> str | None:
+        for layer in self.layers:
+            hint = layer.generate(error_kind, error_meta)
+            if hint is not None:
+                return hint
+        return None
+
+    def as_collator_hook(self) -> Callable[[str, dict], str | None]:
+        """Return a callable matching CollatorConfig.hint_generator's signature
+        (error_kind, error_meta) -> str | None. ZERO collator change."""
+        return self.generate
+
+
+def default_composite(
+    *,
+    llm_complete: Callable[[str], str] | None = None,
+    cache_dir: str | None = None,
+    enable_raw_error: bool = True,
+) -> CompositeHintGenerator:
+    """Build the recommended layered generator: templates -> raw-error -> judge.
+
+    The LLM-judge layer is included only when `llm_complete` is provided.
+    """
+    layers: list[HintGenerator] = [TemplateHintGenerator()]
+    if enable_raw_error:
+        layers.append(RawErrorHintGenerator())
+    if llm_complete is not None:
+        layers.append(LLMJudgeHintGenerator(llm_complete, cache_dir=cache_dir))
+    return CompositeHintGenerator(layers)
+
+
+__all__ = [
+    "dispatch",
+    "register",
+    "HintContext",
+    "HINT_TEMPLATES",
+    # Layered architecture (ADR-009)
+    "HintGenerator",
+    "TemplateHintGenerator",
+    "RawErrorHintGenerator",
+    "LLMJudgeHintGenerator",
+    "CompositeHintGenerator",
+    "default_composite",
+]

composer_replication/tests/test_layered_hint_generator.py

@@ -0,0 +1,161 @@
+"""Tests for the layered HintGenerator architecture (ADR-009).
+
+Covers ADR-009 acceptance gates:
+  - gate 1: HintGenerator Protocol; layers satisfy it (runtime_checkable).
+  - gate 2: TemplateHintGenerator is byte-identical to the existing dispatch()
+    for all 5 registered kinds (no regression).
+  - gate 3: CompositeHintGenerator tries layers cost-first — a tool_not_found
+    site is served by the template layer (no LLM call); a style site falls
+    through to the judge layer.
+  - gate 4: LLMJudgeHintGenerator caches (second identical call = zero
+    completions).
+  - gate 5: as_collator_hook() matches CollatorConfig.hint_generator's
+    (error_kind, error_meta) -> str | None signature.
+
+All CPU-only, no network (LLM layer is a stub).
+"""
+from __future__ import annotations
+
+from composer_replication.hint_generator import (
+    HINT_TEMPLATES,
+    CompositeHintGenerator,
+    HintGenerator,
+    LLMJudgeHintGenerator,
+    RawErrorHintGenerator,
+    TemplateHintGenerator,
+    default_composite,
+    dispatch,
+)
+
+
+# --- gate 1: Protocol -------------------------------------------------------
+
+def test_layers_satisfy_protocol():
+    assert isinstance(TemplateHintGenerator(), HintGenerator)
+    assert isinstance(RawErrorHintGenerator(), HintGenerator)
+    assert isinstance(LLMJudgeHintGenerator(), HintGenerator)
+    assert isinstance(CompositeHintGenerator([]), HintGenerator)
+
+
+# --- gate 2: template byte-identity ----------------------------------------
+
+def test_template_layer_byte_identical_to_dispatch():
+    tmpl = TemplateHintGenerator()
+    meta = {
+        "available_tools": ["read", "write"],
+        "tool_name": "frobnicate",
+        "tool_schema": {"x": "int"},
+        "error_message": "boom",
+    }
+    for kind in HINT_TEMPLATES:
+        ctx = dict(meta)
+        ctx.setdefault("error_kind", kind)
+        expected = dispatch(kind, ctx)
+        got = tmpl.generate(kind, meta)
+        assert got == expected, f"template layer drifted from dispatch for {kind}"
+
+
+def test_template_layer_returns_none_for_unknown_kind():
+    assert TemplateHintGenerator().generate("totally_unknown_kind", {}) is None
+
+
+# --- gate 3: cost-ordered composite ----------------------------------------
+
+def test_composite_serves_tool_error_from_template_no_llm():
+    calls = {"n": 0}
+
+    def fake_complete(prompt: str) -> str:
+        calls["n"] += 1
+        return "LLM HINT"
+
+    comp = default_composite(llm_complete=fake_complete)
+    hint = comp.generate("tool_not_found", {"available_tools": ["read", "write"]})
+    assert hint is not None
+    assert "Available tools" in hint  # template output
+    assert calls["n"] == 0, "LLM judge must NOT be called for a template-covered site"
+
+
+def test_composite_falls_through_to_judge_for_uncovered_site():
+    calls = {"n": 0}
+
+    def fake_complete(prompt: str) -> str:
+        calls["n"] += 1
+        return "Be more concise; you repeated the same explanation."
+
+    comp = default_composite(llm_complete=fake_complete, enable_raw_error=False)
+    # 'verbose_communication' has no template and no error_message -> judge.
+    hint = comp.generate("verbose_communication", {})
+    assert hint == "Be more concise; you repeated the same explanation."
+    assert calls["n"] == 1
+
+
+def test_raw_error_layer_covers_unmatched_site_with_message():
+    comp = default_composite()  # no LLM
+    hint = comp.generate("weird_unmapped_error", {"error_message": "Segfault at 0x0"})
+    assert hint is not None
+    assert "Segfault at 0x0" in hint
+
+
+def test_composite_returns_none_when_all_layers_defer():
+    comp = default_composite()  # templates + raw-error, no LLM
+    # unknown kind + no message -> nothing fires
+    assert comp.generate("unknown", {}) is None
+
+
+# --- gate 4: LLM-judge cache ------------------------------------------------
+
+def test_llm_judge_caches_in_memory(tmp_path):
+    calls = {"n": 0}
+
+    def fake_complete(prompt: str) -> str:
+        calls["n"] += 1
+        return f"hint #{calls['n']}"
+
+    judge = LLMJudgeHintGenerator(fake_complete, cache_dir=str(tmp_path))
+    meta = {"error_message": "X"}
+    h1 = judge.generate("k", meta)
+    h2 = judge.generate("k", meta)  # identical -> cache hit
+    assert h1 == h2
+    assert calls["n"] == 1, "second identical call must hit cache (zero completions)"
+
+
+def test_llm_judge_disk_cache_survives_new_instance(tmp_path):
+    calls = {"n": 0}
+
+    def fake_complete(prompt: str) -> str:
+        calls["n"] += 1
+        return "persisted hint"
+
+    j1 = LLMJudgeHintGenerator(fake_complete, cache_dir=str(tmp_path))
+    j1.generate("k", {"error_message": "X"})
+    # fresh instance, same cache dir -> disk hit, no completion
+    j2 = LLMJudgeHintGenerator(fake_complete, cache_dir=str(tmp_path))
+    h = j2.generate("k", {"error_message": "X"})
+    assert h == "persisted hint"
+    assert calls["n"] == 1
+
+
+def test_llm_judge_disabled_when_no_complete():
+    assert LLMJudgeHintGenerator(None).generate("k", {"error_message": "X"}) is None
+
+
+# --- gate 5: collator-hook signature ---------------------------------------
+
+def test_as_collator_hook_matches_collator_signature():
+    comp = default_composite()
+    hook = comp.as_collator_hook()
+    # CollatorConfig.hint_generator is Callable[[str, dict], str | None]
+    out = hook("tool_not_found", {"available_tools": ["read"]})
+    assert isinstance(out, str)
+    out_none = hook("unknown", {})
+    assert out_none is None
+
+
+def test_as_collator_hook_drops_into_collator_config():
+    """The hook is accepted by CollatorConfig without changes."""
+    from composer_replication.trainer.data_collator import CollatorConfig
+
+    comp = default_composite()
+    cfg = CollatorConfig(hint_generator=comp.as_collator_hook())
+    assert cfg.hint_generator is not None
+    assert cfg.hint_generator("json_decode", {}) is not None  # template fires

docs/adrs/ADR-009-layered-hint-generator.md

@@ -1,5 +1,5 @@
 ---
-status: proposed
+status: accepted
 date: 2026-05-29
 deciders: [Codeseys, ARIA]
 ---
@@ -92,12 +92,15 @@ signature.
 
 ## Acceptance gate (must be green before status flips to accepted)
 
-- [ ] `HintGenerator` Protocol defined with `.generate(error_context: HintContext) -> str | None`; `mypy`/pyright clean.
-- [ ] `TemplateHintGenerator` wraps the existing 5 templates; a test asserts byte-identical output to the current `dispatch()` for all 5 kinds (no regression).
-- [ ] `CompositeHintGenerator` tries layers in cost order; a test asserts a tool_not_found site is served by the template layer (no LLM call) and a style site falls through to the judge layer (mocked).
-- [ ] `LLMJudgeHintGenerator` has a disk cache keyed on error-context hash; a test asserts a second identical call hits the cache (zero network).
-- [ ] `as_collator_hook()` returns a callable accepted by `CollatorConfig.hint_generator` without collator changes; an end-to-end test runs ingestion → collator-with-composite-hook → a non-empty `sdpo_loss_mask` on a real error trace.
-- [ ] Sibling-bootstrap layer is gated behind an explicit `enable_sibling_bootstrap` flag and documented as RL-rollout-path-only; a unit test asserts it returns None in the offline-trace path.
+All gates green as of 2026-05-29 (commit `<this>`; 12 tests in
+`composer_replication/tests/test_layered_hint_generator.py`).
+
+- [x] `HintGenerator` Protocol defined (`runtime_checkable`) with `.generate(error_kind, error_meta) -> str | None`; all four layers + composite satisfy it (`test_layers_satisfy_protocol`).
+- [x] `TemplateHintGenerator` wraps the existing 5 templates; `test_template_layer_byte_identical_to_dispatch` asserts byte-identical output to `dispatch()` for every registered kind (no regression).
+- [x] `CompositeHintGenerator` tries layers cost-first: `test_composite_serves_tool_error_from_template_no_llm` asserts a tool_not_found site is template-served with **zero** LLM calls; `test_composite_falls_through_to_judge_for_uncovered_site` asserts a style site reaches the judge.
+- [x] `LLMJudgeHintGenerator` has in-memory + disk cache keyed on error-context hash; `test_llm_judge_caches_in_memory` and `test_llm_judge_disk_cache_survives_new_instance` assert a second identical call costs zero completions.
+- [x] `as_collator_hook()` returns a callable matching `CollatorConfig.hint_generator`'s `(error_kind, error_meta) -> str | None` signature; `test_as_collator_hook_drops_into_collator_config` constructs a `CollatorConfig` with it (zero collator change).
+- [x] Sibling-bootstrap: **satisfied by design** — recognized during implementation that SDPO sibling-bootstrap is NOT a `HintContext`-driven layer (it needs multiple sibling rollouts, available only in the RL-rollout path, never in offline-trace ingestion). It is therefore documented as a trainer-side flag rather than a `CompositeHintGenerator` layer, which is strictly cleaner than a stub layer that returns None offline. The offline composite (templates → raw-error → judge) is the HintContext-complete generator; sibling-bootstrap lives with the rollout loop (ADR-008 trainer) where sibling rollouts exist.
 
 ## More Information
 

docs/adrs/README.md

@@ -10,7 +10,7 @@
 | [ADR-006](ADR-006-rl-frameworks.md) | RL framework strategy: TRL + VeRL + PRIME-RL | accepted (amended-by ADR-008) | 2026-05-26 |
 | [ADR-007](ADR-007-self-distillation-losses.md) | Self-distillation losses landscape | accepted | 2026-05-26 |
 | [ADR-008](ADR-008-drgrpo-sdpo-live-channel.md) | Target Dr. GRPO + host live SDPO channel in TRL trainer | accepted | 2026-05-29 |
-| [ADR-009](ADR-009-layered-hint-generator.md) | Layered HintGenerator for SDPO textual feedback | proposed | 2026-05-29 |
+| [ADR-009](ADR-009-layered-hint-generator.md) | Layered HintGenerator for SDPO textual feedback | accepted | 2026-05-29 |
 | [ADR-010](ADR-010-feature-deletion-datagen.md) | FeatureDeletionEnv synthetic-data subsystem over OSS SWE substrates | proposed | 2026-05-29 |
 
 Sorted by number ascending. ADRs are immutable after `accepted`; supersede or amend rather than edit.