Spaces:
Running on Zero
feat: implement Archive feature for past session management
Browse files- Added `archive.py` to handle listing and loading past runs for a specific user and scenario.
- Introduced `list_runs` function to retrieve past runs scoped by user session ID and scenario.
- Implemented `load_replay` function to create a read-only `ReplaySession` for a given run ID.
- Created helper functions for formatting run cards and timestamps.
- Enhanced CSS styles for the Archive drawer and run cards to improve UI presentation.
- Updated `FishbowlSession` and `ReplaySession` classes to support new functionality and ensure session isolation.
- Added comprehensive tests for the Archive feature, ensuring correct session attribution and replay functionality.
- Modified memory index tests to include run ID scoping for event searches.
- Created new tests for run history and lifecycle management, ensuring proper event emission and session handling.
- docs/adr/0026-run-lifecycle-and-history.md +135 -0
- docs/adr/0027-per-user-sessions-and-archive-replay.md +77 -0
- docs/adr/0028-session-stamped-events-and-scoped-recall.md +61 -0
- src/core/conductor.py +119 -22
- src/core/events.py +49 -0
- src/core/ledger.py +11 -0
- src/core/memory.py +16 -3
- src/core/memory_index.py +40 -12
- src/core/projections.py +3 -1
- src/core/run_index.py +135 -0
- src/core/sqlalchemy_ledger.py +24 -0
- src/core/sqlite_ledger.py +46 -35
- src/ui/fishbowl/app.py +173 -1
- src/ui/fishbowl/archive.py +145 -0
- src/ui/fishbowl/assets/styles.css +82 -0
- src/ui/fishbowl/session.py +133 -10
- tests/test_fishbowl_archive.py +244 -0
- tests/test_memory.py +1 -1
- tests/test_memory_index.py +21 -2
- tests/test_run_history.py +342 -0
|
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ADR-0026: Run Lifecycle, Provenance, and Multi-Run History in One Ledger
|
| 2 |
+
|
| 3 |
+
## Status
|
| 4 |
+
|
| 5 |
+
Accepted
|
| 6 |
+
|
| 7 |
+
## Context
|
| 8 |
+
|
| 9 |
+
The engine has always known how a run *begins* β `Conductor.reset()` mints a `run_id`
|
| 10 |
+
and appends `run.started` β but it never recorded how a run *ends*, and `reset()` itself
|
| 11 |
+
was destructive in a way that fights the persistent store. Three gaps:
|
| 12 |
+
|
| 13 |
+
1. **No terminal event.** A run stopped for many reasons β the judge reached a
|
| 14 |
+
`judge.verdict`, the governor tripped (`BudgetExceeded`, ADR-0013), the tick cap was
|
| 15 |
+
hit, or the operator pressed stop β but the ledger held no record of *which*, nor a
|
| 16 |
+
tidy summary (who won, on what model, how many turns/tokens). Every consumer (UI,
|
| 17 |
+
trace export, resume) had to re-derive "is this run over and why" from the tail.
|
| 18 |
+
2. **`reset()` wiped the database.** `reset()` called `self.ledger.reset()`, which on a
|
| 19 |
+
persistent backend (`make_ledger()` always returns `SqlAlchemyLedger`; the Fishbowl
|
| 20 |
+
session already uses it) erased *every* prior run. Starting a second show silently
|
| 21 |
+
destroyed the first β directly at odds with wanting a durable, shareable history.
|
| 22 |
+
3. **Thin provenance + run-blind projections.** `run.started` carried only `{seed, goal}`,
|
| 23 |
+
so a replay could not tell which scenario ran or which model each agent was bound to
|
| 24 |
+
(ADR-0022's per-agent binding was invisible after the fact). And projections/memory
|
| 25 |
+
rebuilt from *all* ledger events, so they could not be scoped to a single run once the
|
| 26 |
+
DB held many.
|
| 27 |
+
|
| 28 |
+
The hackathon bar wants the append-only ledger to *be* the shareable agent trace
|
| 29 |
+
(Sharing-is-Caring badge). A trace you can't bound to one run, that loses its own start
|
| 30 |
+
provenance, and that gets clobbered on the next Summon is not shareable.
|
| 31 |
+
|
| 32 |
+
## Decision
|
| 33 |
+
|
| 34 |
+
Treat the ledger as a **multi-run history**: one durable store holds many runs, each
|
| 35 |
+
framed by an enriched `run.started` and a new `run.finished`, queryable per run. All
|
| 36 |
+
changes are additive β `schema_version` stays **1** (ADR-0009 open/additive kinds), no
|
| 37 |
+
migration.
|
| 38 |
+
|
| 39 |
+
- **New terminal kind `run.finished`** (added to `CORE_EVENT_KINDS`, regex-valid under
|
| 40 |
+
ADR-0009). Payload shape:
|
| 41 |
+
|
| 42 |
+
```
|
| 43 |
+
{
|
| 44 |
+
"reason": "verdict" | "budget" | "tick_cap" | "user_stop",
|
| 45 |
+
"winner": str | None, # actor name of the winner, if any
|
| 46 |
+
"winning_model": str | None, # model_endpoint bound to the winner, if known
|
| 47 |
+
"turns": int, # turns elapsed in the run
|
| 48 |
+
"tokens": int # total tokens spent in the run
|
| 49 |
+
}
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
`winner`/`winning_model` are populated from a `judge.verdict` when the reason is
|
| 53 |
+
`verdict` (the winner's model is resolved through the castβmodel map below); both are
|
| 54 |
+
`null` for budget/tick-cap/user-stop endings. `turns`/`tokens` are read from the
|
| 55 |
+
governor's live stats (`src/core/governor.py`).
|
| 56 |
+
|
| 57 |
+
- **Enriched `run.started` payload.** Keep the existing `{seed, goal}` and add, alongside
|
| 58 |
+
them, the **scenario name** and a **castβmodel binding map** (ADR-0022 made visible
|
| 59 |
+
*after* the fact):
|
| 60 |
+
|
| 61 |
+
```
|
| 62 |
+
{
|
| 63 |
+
"seed": str, "goal": str, # existing β unchanged
|
| 64 |
+
"scenario": str, # scenario name
|
| 65 |
+
"cast": { # one entry per agent
|
| 66 |
+
"<agent_name>": {"model_endpoint": str | None,
|
| 67 |
+
"model_profile": "tiny"|"fast"|"balanced"|"strong"}
|
| 68 |
+
}
|
| 69 |
+
}
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
The map is built from `self.scenario.agents`, reading each `agent.manifest.model_endpoint`
|
| 73 |
+
/ `model_profile` and guarding agents that lack a manifest (`getattr(agent,"manifest",None)`).
|
| 74 |
+
Purely additive keys, so old readers ignore them and `schema_version` stays 1.
|
| 75 |
+
|
| 76 |
+
- **Non-destructive `reset()`.** On a persistent ledger, `reset()` **no longer wipes the
|
| 77 |
+
DB**. It mints a new `run_id`, clears only the conductor's *in-memory* turn state
|
| 78 |
+
(queues, `agent_errors`, governor, turn counter), and appends a fresh enriched
|
| 79 |
+
`run.started` plus the scenario genesis. Prior runs stay on disk, so one DB accumulates
|
| 80 |
+
a history of shows. (The in-memory `Ledger` used by some tests is naturally fresh per
|
| 81 |
+
instance; the destructive `ledger.reset()` call is dropped from the lifecycle.)
|
| 82 |
+
|
| 83 |
+
- **Run-scoped projections (optional `run_id`).** `rebuild_stage(events, run_id=None)` and
|
| 84 |
+
`EpisodicMemory/SalienceMemory.visible(events, run_id=None)` gain an optional `run_id`
|
| 85 |
+
that, when given, filters the tuple to that run before projecting β trivial because
|
| 86 |
+
every `Event` carries `run_id`. Default `None` preserves today's whole-ledger behaviour,
|
| 87 |
+
so existing callers and tests are untouched.
|
| 88 |
+
|
| 89 |
+
- **New ledger query API.** Add `events_for_run(run_id) -> tuple[Event, ...]` and
|
| 90 |
+
`runs() -> list[...]` (one descriptor per `run.started`, newest first, with id + scenario
|
| 91 |
+
+ start time) to the ledger surface. Both SQL backends already index `run_id`, so
|
| 92 |
+
`events_for_run` is an indexed query mirroring the existing `tail(from_offset)` /
|
| 93 |
+
`latest_offset()` style; the in-memory backend filters its `events` tuple.
|
| 94 |
+
|
| 95 |
+
## Consequences
|
| 96 |
+
|
| 97 |
+
- **The trace is self-describing and bounded.** A single run.startedββ¦βrun.finished slice
|
| 98 |
+
names its scenario, its full castβmodel binding, how it ended, who won on what model, and
|
| 99 |
+
its turn/token cost β ready to export to the HF Hub as a stand-alone trace.
|
| 100 |
+
- **History survives.** Summoning a new show no longer destroys prior runs; one DB is a
|
| 101 |
+
gallery of shows. The UI can list `runs()` and replay any one via `events_for_run`.
|
| 102 |
+
- **Run-scoped views, no double counting.** Stage and memory can be rebuilt for a chosen
|
| 103 |
+
run even when the store holds many, so a multi-run DB doesn't bleed one show's lines into
|
| 104 |
+
another's blackboard.
|
| 105 |
+
- **Governor stops are recorded, not just raised.** `BudgetExceeded` (ADR-0013) still
|
| 106 |
+
bubbles, but the lifecycle now also writes a `run.finished{reason:"budget"}` so the stop
|
| 107 |
+
is visible in the trace rather than inferred from an absent next event.
|
| 108 |
+
- **Additive only.** No schema bump, no migration; old ledgers and old readers keep
|
| 109 |
+
working, and the deterministic offline stub stays byte-reproducible.
|
| 110 |
+
- **Garbage collection is out of scope.** An ever-growing single DB will eventually want
|
| 111 |
+
pruning/archival of old runs; deferred until the history is large enough to matter.
|
| 112 |
+
|
| 113 |
+
## Alternatives considered
|
| 114 |
+
|
| 115 |
+
- **Keep wiping on reset, one run per DB.** Simplest, but loses all history on Summon and
|
| 116 |
+
makes a shareable multi-run trace impossible β the exact problem this ADR fixes.
|
| 117 |
+
- **Separate DB file per run.** Preserves isolation, but fragments the history, complicates
|
| 118 |
+
`runs()`/replay, and breaks the "one append-only ledger is the trace" story. Rejected for
|
| 119 |
+
a single run-scoped store.
|
| 120 |
+
- **Derive end-reason/winner at read time instead of a `run.finished` event.** Keeps the
|
| 121 |
+
ledger thinner, but every consumer re-implements the heuristic and a budget/user stop
|
| 122 |
+
leaves no positive marker. An explicit terminal event is the append-only-friendly record.
|
| 123 |
+
- **Bump `schema_version` for the richer payloads.** Unnecessary β all additions are new
|
| 124 |
+
keys/kinds, which ADR-0009's open/additive contract already covers at version 1.
|
| 125 |
+
|
| 126 |
+
## References
|
| 127 |
+
|
| 128 |
+
- Builds on ADR-0009 (open/additive event kinds β no migration for new kinds/keys) and
|
| 129 |
+
ADR-0022 (per-agent explicit model binding β the source of the castβmodel map).
|
| 130 |
+
- Relates to ADR-0013 (token governor β supplies the budget stop reason and turn/token
|
| 131 |
+
stats) and ADR-0024 (observability).
|
| 132 |
+
- `src/core/events.py` β `CORE_EVENT_KINDS` (`run.finished`), documented payload shapes
|
| 133 |
+
- `src/core/conductor.py` β non-destructive `reset()`, enriched `run.started`, `run.finished`
|
| 134 |
+
- `src/core/projections.py`, `src/core/memory.py` β optional `run_id` scoping
|
| 135 |
+
- `src/core/ledger.py`, `src/core/sqlalchemy_ledger.py` β `events_for_run`, `runs`
|
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ADR-0027: Per-User Sessions and Read-Only Archive Replay
|
| 2 |
+
|
| 3 |
+
## Status
|
| 4 |
+
|
| 5 |
+
Accepted
|
| 6 |
+
|
| 7 |
+
## Context
|
| 8 |
+
|
| 9 |
+
ADR-0026 made the ledger a durable, multi-run history: many runs live in one store,
|
| 10 |
+
each framed by an enriched `run.started` and a `run.finished`, and the live Conductor
|
| 11 |
+
projection is run-scoped. That unlocked "keep the history of runs" at the data layer β
|
| 12 |
+
but the Fishbowl UI had two gaps that surfaced as soon as the store became shared:
|
| 13 |
+
|
| 14 |
+
1. **The live transcript bled across runs and scenarios.** `FishbowlSession.events`
|
| 15 |
+
returned `conductor.ledger.events` β the *entire* shared ledger β and `snapshot()`
|
| 16 |
+
fed that unscoped into `view_model_at` β `rebuild_stage`. With one durable store
|
| 17 |
+
holding every run, the Show for scenario B would replay scenario A's discussion.
|
| 18 |
+
This is the "show the right discussion for the right session/scenario" bug.
|
| 19 |
+
2. **No notion of "whose run".** Runs were not attributed to a user/browser, so there
|
| 20 |
+
was no basis for "show me *my* past sessions" β and no UI to load them. Summon
|
| 21 |
+
already started a fresh run (ADR-0026's non-destructive reset), but there was no way
|
| 22 |
+
back to a previous one.
|
| 23 |
+
|
| 24 |
+
The product ask: Summon always starts a fresh conversation; previous sessions appear
|
| 25 |
+
only behind a **Load** affordance; each user/browser gets a unique session id; and the
|
| 26 |
+
Archive lists *my* sessions for *this* world only.
|
| 27 |
+
|
| 28 |
+
## Decision
|
| 29 |
+
|
| 30 |
+
Attribute runs to a per-browser session id, scope every Show read to a single run, and
|
| 31 |
+
add a read-only **Archive** that replays a past run without spending tokens. All
|
| 32 |
+
additive; `schema_version` stays **1** (ADR-0009).
|
| 33 |
+
|
| 34 |
+
- **`run.started.session_id` (optional).** `Conductor.reset(seed, *, session_id=None)`
|
| 35 |
+
stamps the id onto `run.started` when present (key omitted otherwise). `RunSummary`
|
| 36 |
+
(ADR-0026) carries `session_id` so the run-index folds it for free. No side table β
|
| 37 |
+
the run list stays a projection of the log (ADR-0014).
|
| 38 |
+
|
| 39 |
+
- **The session id is the browser's, persisted in `localStorage`.** Resolved (or minted
|
| 40 |
+
with `crypto.randomUUID`) by a small JS function on `demo.load`, written into a hidden
|
| 41 |
+
carrier, and read by Python. It survives reloads, so "my sessions" is stable per user
|
| 42 |
+
without server-side accounts.
|
| 43 |
+
|
| 44 |
+
- **Run-scoped live transcript.** `FishbowlSession.events` now returns
|
| 45 |
+
`events_for_run(conductor.run_id)`. Every downstream read (`head`, `snapshot`, the
|
| 46 |
+
scrubber, autoplay) flows from this, so scoping one property scopes the whole Show β
|
| 47 |
+
scenario B can never show scenario A's discussion.
|
| 48 |
+
|
| 49 |
+
- **`ReplaySession` β a read-only view over one past run.** It exposes the exact read
|
| 50 |
+
surface the Show renders (`events`/`head`/`snapshot`/`has_verdict`) over a fixed
|
| 51 |
+
`events_for_run(run_id)` slice, rebuilding cast/meters from the run's own scenario
|
| 52 |
+
(named on `run.started`). `replay = True`; `step`/`step_one`/`inject` are no-ops, and
|
| 53 |
+
the autoplay loop replays the recorded prefix then stops β **a load never generates.**
|
| 54 |
+
|
| 55 |
+
- **`src/ui/fishbowl/archive.py`.** `list_runs(scenario, session_id)` folds the ledger
|
| 56 |
+
via `index_runs_from_ledger` and keeps only this user's runs in this world, newest
|
| 57 |
+
first; `load_replay(run_id)` builds the `ReplaySession`; `run_card_label` renders a
|
| 58 |
+
one-line phosphor card.
|
| 59 |
+
|
| 60 |
+
- **Archive drawer in the Lab.** A `gr.Accordion` under Summon with a `gr.render` keyed
|
| 61 |
+
on (scenario, session id) that lists clickable cards; a card loads the replay and
|
| 62 |
+
jumps to the Show. The render re-runs on world change, when the id resolves from
|
| 63 |
+
localStorage, and on manual refresh.
|
| 64 |
+
|
| 65 |
+
## Consequences
|
| 66 |
+
|
| 67 |
+
- The Show is correct under a shared, multi-user store: each user drives their own
|
| 68 |
+
`run_id` in their own `gr.State`, sees only their own live run, and can browse only
|
| 69 |
+
their own history. Multiple browsers naturally share the store while staying isolated.
|
| 70 |
+
- The Archive is "nearly free" on top of ADR-0026 β pure reads, no new persistence.
|
| 71 |
+
- `ReplaySession` cannot resume a past run (read-only by design); resuming would need a
|
| 72 |
+
live Conductor rehydrated at that `run_id` (a future step if wanted).
|
| 73 |
+
- localStorage ties "my sessions" to a browser profile; clearing storage starts a new
|
| 74 |
+
identity. Acceptable for a no-accounts demo; a future signed-in id could supersede it.
|
| 75 |
+
- Postgres is unchanged β no new tables β keeping the ledger the single source of truth
|
| 76 |
+
(ADR-0014). A materialized run-summary table remains an option if listing ever needs
|
| 77 |
+
to scale beyond folding the log.
|
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ADR-0028: Session-Stamped Events and Run-Scoped Memory Recall
|
| 2 |
+
|
| 3 |
+
## Status
|
| 4 |
+
|
| 5 |
+
Accepted
|
| 6 |
+
|
| 7 |
+
## Context
|
| 8 |
+
|
| 9 |
+
ADR-0027 attributed *runs* to a browser session via `run.started.payload.session_id`,
|
| 10 |
+
and scoped the **UI** to one run. Three gaps remained on the data/context side:
|
| 11 |
+
|
| 12 |
+
1. **Actions were not attributable.** Only `run.started` carried the session id; the
|
| 13 |
+
events an agent actually produced (`agent.spoke`, `judge.verdict`, β¦) did not, so
|
| 14 |
+
"all actions by user X" required a join through `run.started` in every consumer
|
| 15 |
+
(SQL, mem0, exports).
|
| 16 |
+
2. **Agent context bled across runs.** `Conductor._run_agent` passed the *whole*
|
| 17 |
+
ledger (`ledger.events` β every run, every user) as `recent_events`, so episodic
|
| 18 |
+
and salience memory could recall another show's β another user's β discussion
|
| 19 |
+
into a prompt.
|
| 20 |
+
3. **Semantic recall was unscoped.** The mem0 index (ADR-0018/0019/0020) stored and
|
| 21 |
+
searched one global namespace (`user_id="ledger"`), ignoring mem0's native
|
| 22 |
+
`run_id`/`agent_id` identity scopes; cross-run hits crowded the relevance budget.
|
| 23 |
+
|
| 24 |
+
Additionally, the session id originates in `localStorage` β untrusted client input β
|
| 25 |
+
and reached the ledger unvalidated.
|
| 26 |
+
|
| 27 |
+
## Decision
|
| 28 |
+
|
| 29 |
+
- **`Event.session_id: str | None` on the envelope.** Stamped by the Conductor at
|
| 30 |
+
the single `_append` chokepoint from the run's normalized session id; agents and
|
| 31 |
+
scenarios never know sessions exist. Nullable (headless runs stay `None`).
|
| 32 |
+
Persisted as an indexed nullable column in both SQL backends. Additive β
|
| 33 |
+
`schema_version` stays 1. *No migration shipped:* we are pre-release; recreate dev
|
| 34 |
+
DB files instead.
|
| 35 |
+
- **Normalize at the engine boundary.** `normalize_session_id()` (events.py) accepts
|
| 36 |
+
`[A-Za-z0-9._-]{1,64}` and degrades anything else to `None` (logged) β applied in
|
| 37 |
+
`Conductor.reset` (write path) and `archive.list_runs` (read path). A tampered
|
| 38 |
+
localStorage can never break Summon or inject garbage into the store.
|
| 39 |
+
- **Run-scoped agent context.** `_run_agent` now passes
|
| 40 |
+
`events_for_run(self.run_id)` as `recent_events`. One line; closes the prompt
|
| 41 |
+
bleed for both memory layers, since every recall folds from that slice.
|
| 42 |
+
- **mem0 native scoping.** `MemoryIndex.search(query, k, run_id=None)`; backends
|
| 43 |
+
store with mem0's native `run_id=event.run_id` / `agent_id=event.actor` and filter
|
| 44 |
+
search by `run_id`, with a defensive post-filter on the reconstructed event.
|
| 45 |
+
`SalienceMemory` derives the scope from its candidates (single-run slice β scoped
|
| 46 |
+
search, free for callers). `session_id` rides in index metadata for forensics.
|
| 47 |
+
- **RunIndex prefers the envelope.** `RunSummary.session_id` folds from
|
| 48 |
+
`event.session_id` first, payload copy second.
|
| 49 |
+
|
| 50 |
+
## Consequences
|
| 51 |
+
|
| 52 |
+
- Every action is directly filterable by session in SQL (`WHERE session_id = ?`,
|
| 53 |
+
indexed), in mem0 metadata, and in exported traces (the JSONL dump inherits the
|
| 54 |
+
envelope field for free).
|
| 55 |
+
- Prompts are hermetic per run: neither the episodic window, nor salience ranking,
|
| 56 |
+
nor semantic recall can surface another run's text. Verified by probe-agent and
|
| 57 |
+
scoped-search tests.
|
| 58 |
+
- Existing dev databases predate the `session_id` column and must be deleted (or
|
| 59 |
+
recreated) β accepted in lieu of a migration while pre-release.
|
| 60 |
+
- mem0's `_indexed` dedup set is process-local; a persistent vector store may
|
| 61 |
+
re-embed after restart (idempotent by `event.id`, so correct β just re-work).
|
|
@@ -39,7 +39,7 @@ from typing import TYPE_CHECKING
|
|
| 39 |
from uuid import uuid4
|
| 40 |
|
| 41 |
from src import observability as obs
|
| 42 |
-
from src.core.events import Event
|
| 43 |
from src.core.governor import BudgetExceeded, Governor
|
| 44 |
from src.core.ledger import Ledger
|
| 45 |
from src.core.projections import StageProjection, rebuild_stage
|
|
@@ -69,6 +69,9 @@ class Conductor:
|
|
| 69 |
self.snapshot_every = snapshot_every
|
| 70 |
self.snapshot_path = snapshot_path
|
| 71 |
self.run_id = str(uuid4())
|
|
|
|
|
|
|
|
|
|
| 72 |
self.turn = 0
|
| 73 |
self._trigger_queue: deque[tuple["Agent", Event]] = deque()
|
| 74 |
# Actors still to act in the CURRENT turn β the queue ``step_one`` drains one
|
|
@@ -84,33 +87,109 @@ class Conductor:
|
|
| 84 |
|
| 85 |
@property
|
| 86 |
def projection(self) -> StageProjection:
|
| 87 |
-
|
|
|
|
|
|
|
| 88 |
|
| 89 |
# ββ lifecycle βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 90 |
|
| 91 |
-
def
|
| 92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 93 |
if self.observer:
|
| 94 |
self.observer.reset()
|
| 95 |
self._trigger_queue.clear()
|
| 96 |
self._pending.clear()
|
| 97 |
self.agent_errors.clear()
|
| 98 |
self.run_id = str(uuid4())
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
self.turn = 0
|
| 100 |
self.governor.reset()
|
|
|
|
|
|
|
|
|
|
| 101 |
obs.set_context(run_id=self.run_id, turn=self.turn)
|
| 102 |
-
obs.log("run.started", run_id=self.run_id, seed=seed, goal=
|
|
|
|
|
|
|
|
|
|
| 103 |
genesis_start = Event(
|
| 104 |
run_id=self.run_id,
|
| 105 |
turn=self.turn,
|
| 106 |
kind="run.started",
|
| 107 |
actor="conductor",
|
| 108 |
-
payload=
|
| 109 |
)
|
| 110 |
self._append(genesis_start)
|
| 111 |
for event in self.scenario.genesis(self.run_id, self.turn, seed):
|
| 112 |
self._append(event)
|
| 113 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 114 |
def restore(self) -> bool:
|
| 115 |
"""Resume a persisted run: adopt the ledger's run_id and last turn.
|
| 116 |
|
|
@@ -134,10 +213,16 @@ class Conductor:
|
|
| 134 |
With an empty ledger, the first tick performs genesis instead of acting
|
| 135 |
(preserving the original auto-reset behaviour)."""
|
| 136 |
for _ in range(max(1, n_ticks)):
|
| 137 |
-
if not self.ledger.
|
| 138 |
self.reset(self.scenario.default_seed)
|
| 139 |
continue
|
| 140 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
self._maybe_snapshot()
|
| 142 |
|
| 143 |
def step_one(self) -> bool:
|
|
@@ -154,23 +239,27 @@ class Conductor:
|
|
| 154 |
|
| 155 |
Returns True when it produced an event (or performed genesis), False when the
|
| 156 |
opened turn had no actors. May raise :class:`BudgetExceeded` like ``step``."""
|
| 157 |
-
if not self.ledger.
|
| 158 |
self.reset(self.scenario.default_seed)
|
| 159 |
return True
|
| 160 |
|
| 161 |
-
|
| 162 |
-
self.turn += 1
|
| 163 |
-
self.governor.begin_turn(self.turn)
|
| 164 |
-
self.governor.check(self.turn)
|
| 165 |
-
obs.set_context(turn=self.turn)
|
| 166 |
-
self._pending.extend(agent for agent, _ in self._trigger_queue)
|
| 167 |
-
self._trigger_queue.clear()
|
| 168 |
-
self._pending.extend(self._tick_scheduled_agents())
|
| 169 |
if not self._pending:
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
# Absorb subscribers this agent's event just triggered into the current turn,
|
| 175 |
# so a subscription cascade still resolves within the turn (as in ``_tick``).
|
| 176 |
while self._trigger_queue:
|
|
@@ -224,7 +313,10 @@ class Conductor:
|
|
| 224 |
run_id=self.run_id,
|
| 225 |
turn=self.turn,
|
| 226 |
projection=projection,
|
| 227 |
-
|
|
|
|
|
|
|
|
|
|
| 228 |
)
|
| 229 |
except BudgetExceeded:
|
| 230 |
raise # an intentional stop from the governor β never swallow it
|
|
@@ -261,6 +353,11 @@ class Conductor:
|
|
| 261 |
snapshot_to(self.snapshot_path)
|
| 262 |
|
| 263 |
def _append(self, event: Event) -> Event:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 264 |
appended = self.ledger.append(event)
|
| 265 |
obs.log(
|
| 266 |
"event.append", level="debug", id=appended.id, kind=appended.kind, actor=appended.actor, turn=appended.turn
|
|
|
|
| 39 |
from uuid import uuid4
|
| 40 |
|
| 41 |
from src import observability as obs
|
| 42 |
+
from src.core.events import Event, normalize_session_id
|
| 43 |
from src.core.governor import BudgetExceeded, Governor
|
| 44 |
from src.core.ledger import Ledger
|
| 45 |
from src.core.projections import StageProjection, rebuild_stage
|
|
|
|
| 69 |
self.snapshot_every = snapshot_every
|
| 70 |
self.snapshot_path = snapshot_path
|
| 71 |
self.run_id = str(uuid4())
|
| 72 |
+
# The browser/user session driving the current run (normalized, untrusted
|
| 73 |
+
# input) β stamped onto every event this conductor appends (see _append).
|
| 74 |
+
self.session_id: str | None = None
|
| 75 |
self.turn = 0
|
| 76 |
self._trigger_queue: deque[tuple["Agent", Event]] = deque()
|
| 77 |
# Actors still to act in the CURRENT turn β the queue ``step_one`` drains one
|
|
|
|
| 87 |
|
| 88 |
@property
|
| 89 |
def projection(self) -> StageProjection:
|
| 90 |
+
# Run-scoped: the live stage shows only the current run, even though the
|
| 91 |
+
# ledger is a shared, append-only store of every run (ADR-0009).
|
| 92 |
+
return rebuild_stage(self.ledger.events, self.run_id)
|
| 93 |
|
| 94 |
# ββ lifecycle βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 95 |
|
| 96 |
+
def _cast_map(self) -> dict[str, dict[str, str | None]]:
|
| 97 |
+
"""Snapshot of each agent's model binding, keyed by agent name.
|
| 98 |
+
|
| 99 |
+
Recorded on ``run.started`` so a run is self-describing β the trace alone
|
| 100 |
+
says which models played which parts (handy for sponsor-track receipts).
|
| 101 |
+
Agents without a manifest (Phase-0/1 fallback) are reported as unbound.
|
| 102 |
+
"""
|
| 103 |
+
cast: dict[str, dict[str, str | None]] = {}
|
| 104 |
+
for agent in self.scenario.agents:
|
| 105 |
+
name = getattr(agent, "name", agent.__class__.__name__)
|
| 106 |
+
manifest = getattr(agent, "manifest", None)
|
| 107 |
+
cast[name] = {
|
| 108 |
+
"model_endpoint": getattr(manifest, "model_endpoint", None),
|
| 109 |
+
"model_profile": getattr(manifest, "model_profile", None),
|
| 110 |
+
}
|
| 111 |
+
return cast
|
| 112 |
+
|
| 113 |
+
def reset(self, seed: str, *, session_id: str | None = None) -> None:
|
| 114 |
+
# NOTE: we no longer wipe the ledger β it is a shared, persistent, append-only
|
| 115 |
+
# store (ADR-0009), so a reset mints a *new* run rather than destroying prior
|
| 116 |
+
# ones. Only the in-conductor transient state for the old run is cleared.
|
| 117 |
+
#
|
| 118 |
+
# ``session_id`` (optional) attributes the run to the browser/user that started
|
| 119 |
+
# it β stamped onto ``run.started`` so the per-user Archive can list "my runs"
|
| 120 |
+
# without a side table (ADR-0014: every view is a projection of the log).
|
| 121 |
if self.observer:
|
| 122 |
self.observer.reset()
|
| 123 |
self._trigger_queue.clear()
|
| 124 |
self._pending.clear()
|
| 125 |
self.agent_errors.clear()
|
| 126 |
self.run_id = str(uuid4())
|
| 127 |
+
# Normalize at the engine boundary: the id originates client-side
|
| 128 |
+
# (localStorage), so malformed/oversized values degrade to None here
|
| 129 |
+
# rather than reaching the ledger or the memory index.
|
| 130 |
+
self.session_id = normalize_session_id(session_id)
|
| 131 |
self.turn = 0
|
| 132 |
self.governor.reset()
|
| 133 |
+
goal = getattr(self.scenario, "goal", "")
|
| 134 |
+
scenario_name = getattr(self.scenario, "name", type(self.scenario).__name__)
|
| 135 |
+
cast = self._cast_map()
|
| 136 |
obs.set_context(run_id=self.run_id, turn=self.turn)
|
| 137 |
+
obs.log("run.started", run_id=self.run_id, seed=seed, goal=goal, scenario=scenario_name)
|
| 138 |
+
payload: dict = {"seed": seed, "goal": goal, "scenario": scenario_name, "cast": cast}
|
| 139 |
+
if self.session_id:
|
| 140 |
+
payload["session_id"] = self.session_id
|
| 141 |
genesis_start = Event(
|
| 142 |
run_id=self.run_id,
|
| 143 |
turn=self.turn,
|
| 144 |
kind="run.started",
|
| 145 |
actor="conductor",
|
| 146 |
+
payload=payload,
|
| 147 |
)
|
| 148 |
self._append(genesis_start)
|
| 149 |
for event in self.scenario.genesis(self.run_id, self.turn, seed):
|
| 150 |
self._append(event)
|
| 151 |
|
| 152 |
+
def finalize(
|
| 153 |
+
self,
|
| 154 |
+
reason: str,
|
| 155 |
+
*,
|
| 156 |
+
winner: str | None = None,
|
| 157 |
+
winning_model: str | None = None,
|
| 158 |
+
) -> Event | None:
|
| 159 |
+
"""Close the current run with a ``run.finished`` event.
|
| 160 |
+
|
| 161 |
+
Idempotent-safe: if this run already has a ``run.finished`` event we return
|
| 162 |
+
the existing one rather than emitting a duplicate. ``turns`` and ``tokens``
|
| 163 |
+
are read from the governor's live counters.
|
| 164 |
+
"""
|
| 165 |
+
existing = [e for e in self.ledger.events_for_run(self.run_id) if e.kind == "run.finished"]
|
| 166 |
+
if existing:
|
| 167 |
+
return existing[0]
|
| 168 |
+
stats = self.governor.stats
|
| 169 |
+
finished = Event(
|
| 170 |
+
run_id=self.run_id,
|
| 171 |
+
turn=self.turn,
|
| 172 |
+
kind="run.finished",
|
| 173 |
+
actor="conductor",
|
| 174 |
+
payload={
|
| 175 |
+
"reason": reason,
|
| 176 |
+
"winner": winner,
|
| 177 |
+
"winning_model": winning_model,
|
| 178 |
+
"turns": int(stats.get("current_turn", self.turn) or self.turn),
|
| 179 |
+
"tokens": int(stats.get("total_tokens", 0) or 0),
|
| 180 |
+
},
|
| 181 |
+
)
|
| 182 |
+
obs.log(
|
| 183 |
+
"run.finished",
|
| 184 |
+
run_id=self.run_id,
|
| 185 |
+
reason=reason,
|
| 186 |
+
winner=winner,
|
| 187 |
+
winning_model=winning_model,
|
| 188 |
+
turns=finished.payload["turns"],
|
| 189 |
+
tokens=finished.payload["tokens"],
|
| 190 |
+
)
|
| 191 |
+
return self._append(finished)
|
| 192 |
+
|
| 193 |
def restore(self) -> bool:
|
| 194 |
"""Resume a persisted run: adopt the ledger's run_id and last turn.
|
| 195 |
|
|
|
|
| 213 |
With an empty ledger, the first tick performs genesis instead of acting
|
| 214 |
(preserving the original auto-reset behaviour)."""
|
| 215 |
for _ in range(max(1, n_ticks)):
|
| 216 |
+
if not self.ledger.events_for_run(self.run_id):
|
| 217 |
self.reset(self.scenario.default_seed)
|
| 218 |
continue
|
| 219 |
+
try:
|
| 220 |
+
self._tick()
|
| 221 |
+
except BudgetExceeded:
|
| 222 |
+
# Close the run on the ledger before the stop propagates β a headless
|
| 223 |
+
# run that hits a budget bound should still be self-describing.
|
| 224 |
+
self.finalize("budget")
|
| 225 |
+
raise
|
| 226 |
self._maybe_snapshot()
|
| 227 |
|
| 228 |
def step_one(self) -> bool:
|
|
|
|
| 239 |
|
| 240 |
Returns True when it produced an event (or performed genesis), False when the
|
| 241 |
opened turn had no actors. May raise :class:`BudgetExceeded` like ``step``."""
|
| 242 |
+
if not self.ledger.events_for_run(self.run_id):
|
| 243 |
self.reset(self.scenario.default_seed)
|
| 244 |
return True
|
| 245 |
|
| 246 |
+
try:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 247 |
if not self._pending:
|
| 248 |
+
self.turn += 1
|
| 249 |
+
self.governor.begin_turn(self.turn)
|
| 250 |
+
self.governor.check(self.turn)
|
| 251 |
+
obs.set_context(turn=self.turn)
|
| 252 |
+
self._pending.extend(agent for agent, _ in self._trigger_queue)
|
| 253 |
+
self._trigger_queue.clear()
|
| 254 |
+
self._pending.extend(self._tick_scheduled_agents())
|
| 255 |
+
if not self._pending:
|
| 256 |
+
return False
|
| 257 |
+
|
| 258 |
+
agent = self._pending.popleft()
|
| 259 |
+
self._run_agent(agent, self.projection)
|
| 260 |
+
except BudgetExceeded:
|
| 261 |
+
self.finalize("budget")
|
| 262 |
+
raise
|
| 263 |
# Absorb subscribers this agent's event just triggered into the current turn,
|
| 264 |
# so a subscription cascade still resolves within the turn (as in ``_tick``).
|
| 265 |
while self._trigger_queue:
|
|
|
|
| 313 |
run_id=self.run_id,
|
| 314 |
turn=self.turn,
|
| 315 |
projection=projection,
|
| 316 |
+
# Run-scoped: the ledger holds EVERY run (shared store, ADR-0026);
|
| 317 |
+
# an agent's memory/context must never recall another run's β or
|
| 318 |
+
# another user's β discussion.
|
| 319 |
+
recent_events=self.ledger.events_for_run(self.run_id),
|
| 320 |
)
|
| 321 |
except BudgetExceeded:
|
| 322 |
raise # an intentional stop from the governor β never swallow it
|
|
|
|
| 353 |
snapshot_to(self.snapshot_path)
|
| 354 |
|
| 355 |
def _append(self, event: Event) -> Event:
|
| 356 |
+
# Stamp the session onto the envelope at the single append chokepoint, so
|
| 357 |
+
# *every* action in a run is attributable/filterable by who drove it β
|
| 358 |
+
# agents and scenarios never have to know sessions exist.
|
| 359 |
+
if self.session_id and event.session_id is None:
|
| 360 |
+
event = event.model_copy(update={"session_id": self.session_id})
|
| 361 |
appended = self.ledger.append(event)
|
| 362 |
obs.log(
|
| 363 |
"event.append", level="debug", id=appended.id, kind=appended.kind, actor=appended.actor, turn=appended.turn
|
|
@@ -20,9 +20,26 @@ from src import observability as obs
|
|
| 20 |
# A kind is a lowercase, dot-namespaced identifier with at least two segments:
|
| 21 |
# <namespace>.<name> e.g. "agent.spoke", "clue.found", "episode.published"
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
CORE_EVENT_KINDS: frozenset[str] = frozenset(
|
| 24 |
{
|
| 25 |
"run.started",
|
|
|
|
| 26 |
"world.observed",
|
| 27 |
"agent.thought",
|
| 28 |
"agent.spoke",
|
|
@@ -44,6 +61,33 @@ def is_valid_kind(kind: str) -> bool:
|
|
| 44 |
return bool(_KIND_PATTERN.match(kind))
|
| 45 |
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
class Event(BaseModel):
|
| 48 |
model_config = ConfigDict(extra="forbid")
|
| 49 |
|
|
@@ -55,6 +99,11 @@ class Event(BaseModel):
|
|
| 55 |
payload: dict[str, Any]
|
| 56 |
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
| 57 |
schema_version: int = 1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
|
| 59 |
@field_validator("kind")
|
| 60 |
@classmethod
|
|
|
|
| 20 |
# A kind is a lowercase, dot-namespaced identifier with at least two segments:
|
| 21 |
# <namespace>.<name> e.g. "agent.spoke", "clue.found", "episode.published"
|
| 22 |
|
| 23 |
+
# Documented core payload shapes (the schema validates kind *format*, not payload β
|
| 24 |
+
# these are conventions for the engine's own events):
|
| 25 |
+
#
|
| 26 |
+
# run.finished payload:
|
| 27 |
+
# {
|
| 28 |
+
# "reason": "verdict" | "budget" | "tick_cap" | "user_stop",
|
| 29 |
+
# "winner": str | None, # actor name of the winner, if any
|
| 30 |
+
# "winning_model": str | None, # model bound to the winner, if known
|
| 31 |
+
# "turns": int, # turns elapsed in the run
|
| 32 |
+
# "tokens": int, # total tokens spent in the run
|
| 33 |
+
# }
|
| 34 |
+
#
|
| 35 |
+
# run.started payload is being ENRICHED in a later step (scenario name +
|
| 36 |
+
# cast->model map). That change is purely ADDITIVE β new keys alongside the
|
| 37 |
+
# existing ones β so `schema_version` stays 1 (no migration required).
|
| 38 |
+
|
| 39 |
CORE_EVENT_KINDS: frozenset[str] = frozenset(
|
| 40 |
{
|
| 41 |
"run.started",
|
| 42 |
+
"run.finished",
|
| 43 |
"world.observed",
|
| 44 |
"agent.thought",
|
| 45 |
"agent.spoke",
|
|
|
|
| 61 |
return bool(_KIND_PATTERN.match(kind))
|
| 62 |
|
| 63 |
|
| 64 |
+
# ββ session ids βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 65 |
+
#
|
| 66 |
+
# A session id attributes a run (and, via the envelope below, every event in it)
|
| 67 |
+
# to the browser/user that started it. The value originates client-side
|
| 68 |
+
# (localStorage), so it is UNTRUSTED input: normalize at the engine boundary
|
| 69 |
+
# before it ever reaches the ledger or the memory index.
|
| 70 |
+
|
| 71 |
+
_SESSION_ID_PATTERN = re.compile(r"^[A-Za-z0-9._-]{1,64}$")
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
def normalize_session_id(value: str | None) -> str | None:
|
| 75 |
+
"""Return a safe session id, or ``None`` when *value* is absent or malformed.
|
| 76 |
+
|
| 77 |
+
Accepts the ids we mint (UUIDs, ``sess-β¦``) and rejects anything else β
|
| 78 |
+
over-long strings, whitespace, control characters, separators that could
|
| 79 |
+
confuse downstream filters. Rejection degrades to an unattributed run
|
| 80 |
+
rather than an error, so a tampered localStorage never breaks Summon.
|
| 81 |
+
"""
|
| 82 |
+
candidate = (value or "").strip()
|
| 83 |
+
if not candidate:
|
| 84 |
+
return None
|
| 85 |
+
if _SESSION_ID_PATTERN.match(candidate):
|
| 86 |
+
return candidate
|
| 87 |
+
obs.log("session.id_rejected", level="warning", length=len(candidate))
|
| 88 |
+
return None
|
| 89 |
+
|
| 90 |
+
|
| 91 |
class Event(BaseModel):
|
| 92 |
model_config = ConfigDict(extra="forbid")
|
| 93 |
|
|
|
|
| 99 |
payload: dict[str, Any]
|
| 100 |
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
| 101 |
schema_version: int = 1
|
| 102 |
+
# Who (which browser/user session) drove the run this event belongs to.
|
| 103 |
+
# Stamped by the Conductor on append (single chokepoint), nullable for
|
| 104 |
+
# headless/legacy events. An OPTIONAL envelope field β additive, so
|
| 105 |
+
# schema_version stays 1 and old rows load with session_id=None.
|
| 106 |
+
session_id: str | None = None
|
| 107 |
|
| 108 |
@field_validator("kind")
|
| 109 |
@classmethod
|
|
@@ -34,3 +34,14 @@ class Ledger:
|
|
| 34 |
obs.log("ledger.reset", level="debug", events=len(self._events))
|
| 35 |
self._events.clear()
|
| 36 |
self._seen_ids.clear()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
obs.log("ledger.reset", level="debug", events=len(self._events))
|
| 35 |
self._events.clear()
|
| 36 |
self._seen_ids.clear()
|
| 37 |
+
|
| 38 |
+
def events_for_run(self, run_id: str) -> tuple[Event, ...]:
|
| 39 |
+
"""Return the events of *run_id* in append/offset order."""
|
| 40 |
+
return tuple(e for e in self._events if e.run_id == run_id)
|
| 41 |
+
|
| 42 |
+
def runs(self) -> tuple[str, ...]:
|
| 43 |
+
"""Return the distinct run_ids in first-seen order."""
|
| 44 |
+
seen: dict[str, None] = {}
|
| 45 |
+
for e in self._events:
|
| 46 |
+
seen.setdefault(e.run_id, None)
|
| 47 |
+
return tuple(seen)
|
|
@@ -88,7 +88,9 @@ class EpisodicMemory:
|
|
| 88 |
agent_name: str
|
| 89 |
max_recent: int = 8
|
| 90 |
|
| 91 |
-
def visible(self, events: tuple[Event, ...]) -> list[Event]:
|
|
|
|
|
|
|
| 92 |
result = [e for e in events if e.actor == self.agent_name or e.kind in _GLOBALLY_VISIBLE]
|
| 93 |
return result[-self.max_recent :]
|
| 94 |
|
|
@@ -181,12 +183,19 @@ class SalienceMemory:
|
|
| 181 |
"""
|
| 182 |
if self.index is None or not query or not candidates:
|
| 183 |
return None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 184 |
# The index is a derived, rebuildable lens (ADR-0018) β never load-bearing.
|
| 185 |
# If it hiccups (a flaky hosted backend, a transient mem0 error), degrade to
|
| 186 |
# keyword relevance rather than let one agent's recall crash its whole turn.
|
| 187 |
try:
|
| 188 |
self.index.index(tuple(candidates))
|
| 189 |
-
hits = self.index.search(query, k=len(candidates))
|
| 190 |
except Exception as exc: # noqa: BLE001 β relevance is best-effort, never fatal
|
| 191 |
logger.warning("memory index unavailable, using keyword relevance: %s", exc)
|
| 192 |
obs.log("memory.index.fallback", level="warning", agent=self.agent_name, error=str(exc))
|
|
@@ -198,7 +207,11 @@ class SalienceMemory:
|
|
| 198 |
n = len(ranked)
|
| 199 |
return {eid: (n - i) / n for i, eid in enumerate(ranked)}
|
| 200 |
|
| 201 |
-
def visible(
|
|
|
|
|
|
|
|
|
|
|
|
|
| 202 |
candidates = self._candidates(events)
|
| 203 |
relevance = self._relevance_map(candidates, query)
|
| 204 |
scored = sorted(
|
|
|
|
| 88 |
agent_name: str
|
| 89 |
max_recent: int = 8
|
| 90 |
|
| 91 |
+
def visible(self, events: tuple[Event, ...], run_id: str | None = None) -> list[Event]:
|
| 92 |
+
if run_id is not None:
|
| 93 |
+
events = tuple(e for e in events if e.run_id == run_id)
|
| 94 |
result = [e for e in events if e.actor == self.agent_name or e.kind in _GLOBALLY_VISIBLE]
|
| 95 |
return result[-self.max_recent :]
|
| 96 |
|
|
|
|
| 183 |
"""
|
| 184 |
if self.index is None or not query or not candidates:
|
| 185 |
return None
|
| 186 |
+
# Scope the semantic search to the candidates' run: the index spans every
|
| 187 |
+
# run in the shared store, and unscoped hits from other runs (other users'
|
| 188 |
+
# shows) would crowd the recall budget out of this run's events. Derived
|
| 189 |
+
# from the candidates so callers that already pass a single-run slice (the
|
| 190 |
+
# conductor does) get scoping for free.
|
| 191 |
+
run_ids = {e.run_id for e in candidates}
|
| 192 |
+
run_id = next(iter(run_ids)) if len(run_ids) == 1 else None
|
| 193 |
# The index is a derived, rebuildable lens (ADR-0018) β never load-bearing.
|
| 194 |
# If it hiccups (a flaky hosted backend, a transient mem0 error), degrade to
|
| 195 |
# keyword relevance rather than let one agent's recall crash its whole turn.
|
| 196 |
try:
|
| 197 |
self.index.index(tuple(candidates))
|
| 198 |
+
hits = self.index.search(query, k=len(candidates), run_id=run_id)
|
| 199 |
except Exception as exc: # noqa: BLE001 β relevance is best-effort, never fatal
|
| 200 |
logger.warning("memory index unavailable, using keyword relevance: %s", exc)
|
| 201 |
obs.log("memory.index.fallback", level="warning", agent=self.agent_name, error=str(exc))
|
|
|
|
| 207 |
n = len(ranked)
|
| 208 |
return {eid: (n - i) / n for i, eid in enumerate(ranked)}
|
| 209 |
|
| 210 |
+
def visible(
|
| 211 |
+
self, events: tuple[Event, ...], current_turn: int, query: str, run_id: str | None = None
|
| 212 |
+
) -> list[Event]:
|
| 213 |
+
if run_id is not None:
|
| 214 |
+
events = tuple(e for e in events if e.run_id == run_id)
|
| 215 |
candidates = self._candidates(events)
|
| 216 |
relevance = self._relevance_map(candidates, query)
|
| 217 |
scored = sorted(
|
|
@@ -91,8 +91,13 @@ class MemoryIndex(Protocol):
|
|
| 91 |
"""Derive/refresh index entries for *events* (idempotent by ``event.id``)."""
|
| 92 |
...
|
| 93 |
|
| 94 |
-
def search(self, query: str, k: int) -> list[Event]:
|
| 95 |
-
"""Return up to *k* indexed events most semantically relevant to *query*.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
...
|
| 97 |
|
| 98 |
|
|
@@ -134,7 +139,7 @@ class _Mem0BackendBase:
|
|
| 134 |
"""Upsert one event verbatim into *mem* (``infer=False``; ledger is truth)."""
|
| 135 |
raise NotImplementedError
|
| 136 |
|
| 137 |
-
def _query(self, mem: object, query: str, k: int) -> list[dict]:
|
| 138 |
"""Run semantic search on *mem*; return raw hit dicts (carrying metadata)."""
|
| 139 |
raise NotImplementedError
|
| 140 |
|
|
@@ -160,8 +165,13 @@ class _Mem0BackendBase:
|
|
| 160 |
self._store(mem, event)
|
| 161 |
self._indexed.add(event.id)
|
| 162 |
|
| 163 |
-
def search(self, query: str, k: int) -> list[Event]:
|
| 164 |
-
"""Semantic search; map hits back to :class:`Event` via stored metadata.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 165 |
if not query or k <= 0:
|
| 166 |
return []
|
| 167 |
with obs.span(
|
|
@@ -171,10 +181,13 @@ class _Mem0BackendBase:
|
|
| 171 |
started = time.perf_counter()
|
| 172 |
mem = self._memory()
|
| 173 |
events: list[Event] = []
|
| 174 |
-
for hit in self._query(mem, query, k):
|
| 175 |
event = _event_from_metadata(hit.get("metadata"))
|
| 176 |
-
if event is
|
| 177 |
-
|
|
|
|
|
|
|
|
|
|
| 178 |
elapsed_ms = (time.perf_counter() - started) * 1000
|
| 179 |
obs.add_span_attrs(**{"memory.hits": len(events), "memory.latency_ms": round(elapsed_ms, 2)})
|
| 180 |
obs.observe("memory.index.hits", len(events))
|
|
@@ -228,12 +241,20 @@ class Mem0MemoryIndex(_Mem0BackendBase):
|
|
| 228 |
mem.add( # type: ignore[attr-defined]
|
| 229 |
_event_text(event),
|
| 230 |
user_id=self._NAMESPACE,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 231 |
metadata=_event_metadata(event),
|
| 232 |
infer=False, # store verbatim; the ledger, not a model, is truth
|
| 233 |
)
|
| 234 |
|
| 235 |
-
def _query(self, mem: object, query: str, k: int) -> list[dict]:
|
| 236 |
-
|
|
|
|
|
|
|
|
|
|
| 237 |
|
| 238 |
|
| 239 |
# ββ hosted (opt-in) backend ββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
@@ -302,12 +323,17 @@ class Mem0CloudIndex(_Mem0BackendBase):
|
|
| 302 |
mem.add( # type: ignore[attr-defined]
|
| 303 |
[{"role": "user", "content": _event_text(event)}],
|
| 304 |
user_id=self._NAMESPACE,
|
|
|
|
|
|
|
| 305 |
metadata=_event_metadata(event),
|
| 306 |
infer=False,
|
| 307 |
)
|
| 308 |
|
| 309 |
-
def _query(self, mem: object, query: str, k: int) -> list[dict]:
|
| 310 |
-
|
|
|
|
|
|
|
|
|
|
| 311 |
|
| 312 |
|
| 313 |
# ββ metadata round-trip (event β vector entry) ββββββββββββββββββββββββββββββββ
|
|
@@ -324,6 +350,7 @@ def _event_metadata(event: Event) -> dict:
|
|
| 324 |
"payload": event.payload,
|
| 325 |
"created_at": event.created_at.isoformat(),
|
| 326 |
"schema_version": event.schema_version,
|
|
|
|
| 327 |
}
|
| 328 |
|
| 329 |
|
|
@@ -340,6 +367,7 @@ def _event_from_metadata(metadata: dict | None) -> Event | None:
|
|
| 340 |
actor=str(metadata.get("actor", "")),
|
| 341 |
payload=dict(metadata.get("payload") or {}),
|
| 342 |
schema_version=int(metadata.get("schema_version", 1)),
|
|
|
|
| 343 |
)
|
| 344 |
except (KeyError, ValueError, TypeError): # pragma: no cover - defensive
|
| 345 |
return None
|
|
|
|
| 91 |
"""Derive/refresh index entries for *events* (idempotent by ``event.id``)."""
|
| 92 |
...
|
| 93 |
|
| 94 |
+
def search(self, query: str, k: int, run_id: str | None = None) -> list[Event]:
|
| 95 |
+
"""Return up to *k* indexed events most semantically relevant to *query*.
|
| 96 |
+
|
| 97 |
+
When *run_id* is given, hits MUST be scoped to that run β the ledger holds
|
| 98 |
+
every run, and recall across runs would leak one show's (or one user's)
|
| 99 |
+
discussion into another's context.
|
| 100 |
+
"""
|
| 101 |
...
|
| 102 |
|
| 103 |
|
|
|
|
| 139 |
"""Upsert one event verbatim into *mem* (``infer=False``; ledger is truth)."""
|
| 140 |
raise NotImplementedError
|
| 141 |
|
| 142 |
+
def _query(self, mem: object, query: str, k: int, run_id: str | None) -> list[dict]:
|
| 143 |
"""Run semantic search on *mem*; return raw hit dicts (carrying metadata)."""
|
| 144 |
raise NotImplementedError
|
| 145 |
|
|
|
|
| 165 |
self._store(mem, event)
|
| 166 |
self._indexed.add(event.id)
|
| 167 |
|
| 168 |
+
def search(self, query: str, k: int, run_id: str | None = None) -> list[Event]:
|
| 169 |
+
"""Semantic search; map hits back to :class:`Event` via stored metadata.
|
| 170 |
+
|
| 171 |
+
*run_id* scopes recall to one run, filtered both natively (mem0's own
|
| 172 |
+
``run_id`` identity, pushed down to the vector store) and defensively
|
| 173 |
+
here on the reconstructed event β belt and suspenders against backends
|
| 174 |
+
that ignore unknown filters."""
|
| 175 |
if not query or k <= 0:
|
| 176 |
return []
|
| 177 |
with obs.span(
|
|
|
|
| 181 |
started = time.perf_counter()
|
| 182 |
mem = self._memory()
|
| 183 |
events: list[Event] = []
|
| 184 |
+
for hit in self._query(mem, query, k, run_id):
|
| 185 |
event = _event_from_metadata(hit.get("metadata"))
|
| 186 |
+
if event is None:
|
| 187 |
+
continue
|
| 188 |
+
if run_id is not None and event.run_id != run_id:
|
| 189 |
+
continue
|
| 190 |
+
events.append(event)
|
| 191 |
elapsed_ms = (time.perf_counter() - started) * 1000
|
| 192 |
obs.add_span_attrs(**{"memory.hits": len(events), "memory.latency_ms": round(elapsed_ms, 2)})
|
| 193 |
obs.observe("memory.index.hits", len(events))
|
|
|
|
| 241 |
mem.add( # type: ignore[attr-defined]
|
| 242 |
_event_text(event),
|
| 243 |
user_id=self._NAMESPACE,
|
| 244 |
+
# mem0's native identity scopes: run_id partitions recall per run and
|
| 245 |
+
# agent_id per actor, so filtering happens in the vector store rather
|
| 246 |
+
# than post-hoc in Python (mem0 best practice).
|
| 247 |
+
run_id=event.run_id,
|
| 248 |
+
agent_id=event.actor or None,
|
| 249 |
metadata=_event_metadata(event),
|
| 250 |
infer=False, # store verbatim; the ledger, not a model, is truth
|
| 251 |
)
|
| 252 |
|
| 253 |
+
def _query(self, mem: object, query: str, k: int, run_id: str | None) -> list[dict]:
|
| 254 |
+
filters: dict = {"user_id": self._NAMESPACE}
|
| 255 |
+
if run_id:
|
| 256 |
+
filters["run_id"] = run_id
|
| 257 |
+
return _result_items(mem.search(query, top_k=k, filters=filters)) # type: ignore[attr-defined]
|
| 258 |
|
| 259 |
|
| 260 |
# ββ hosted (opt-in) backend ββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
|
|
| 323 |
mem.add( # type: ignore[attr-defined]
|
| 324 |
[{"role": "user", "content": _event_text(event)}],
|
| 325 |
user_id=self._NAMESPACE,
|
| 326 |
+
run_id=event.run_id, # native per-run scope (see local backend)
|
| 327 |
+
agent_id=event.actor or None,
|
| 328 |
metadata=_event_metadata(event),
|
| 329 |
infer=False,
|
| 330 |
)
|
| 331 |
|
| 332 |
+
def _query(self, mem: object, query: str, k: int, run_id: str | None) -> list[dict]:
|
| 333 |
+
filters: dict = {"user_id": self._NAMESPACE}
|
| 334 |
+
if run_id:
|
| 335 |
+
filters["run_id"] = run_id
|
| 336 |
+
return _result_items(mem.search(query, top_k=k, filters=filters)) # type: ignore[attr-defined]
|
| 337 |
|
| 338 |
|
| 339 |
# ββ metadata round-trip (event β vector entry) ββββββββββββββββββββββββββββββββ
|
|
|
|
| 350 |
"payload": event.payload,
|
| 351 |
"created_at": event.created_at.isoformat(),
|
| 352 |
"schema_version": event.schema_version,
|
| 353 |
+
"session_id": event.session_id,
|
| 354 |
}
|
| 355 |
|
| 356 |
|
|
|
|
| 367 |
actor=str(metadata.get("actor", "")),
|
| 368 |
payload=dict(metadata.get("payload") or {}),
|
| 369 |
schema_version=int(metadata.get("schema_version", 1)),
|
| 370 |
+
session_id=metadata.get("session_id") or None,
|
| 371 |
)
|
| 372 |
except (KeyError, ValueError, TypeError): # pragma: no cover - defensive
|
| 373 |
return None
|
|
@@ -43,8 +43,10 @@ class StageProjection:
|
|
| 43 |
self.agent_notes = self.agent_notes[-8:]
|
| 44 |
|
| 45 |
|
| 46 |
-
def rebuild_stage(events: tuple[Event, ...]) -> StageProjection:
|
| 47 |
projection = StageProjection()
|
|
|
|
|
|
|
| 48 |
for event in events:
|
| 49 |
projection.apply(event)
|
| 50 |
return projection
|
|
|
|
| 43 |
self.agent_notes = self.agent_notes[-8:]
|
| 44 |
|
| 45 |
|
| 46 |
+
def rebuild_stage(events: tuple[Event, ...], run_id: str | None = None) -> StageProjection:
|
| 47 |
projection = StageProjection()
|
| 48 |
+
if run_id is not None:
|
| 49 |
+
events = tuple(e for e in events if e.run_id == run_id)
|
| 50 |
for event in events:
|
| 51 |
projection.apply(event)
|
| 52 |
return projection
|
|
@@ -0,0 +1,135 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Run index β a per-run summary projection over the append-only ledger.
|
| 2 |
+
|
| 3 |
+
This is a *projection* in the same spirit as :func:`src.core.projections.rebuild_stage`:
|
| 4 |
+
a pure fold over events that yields one :class:`RunSummary` per ``run_id``. It reads the
|
| 5 |
+
two engine bookend events β ``run.started`` (scenario / seed / cast / start time) and
|
| 6 |
+
``run.finished`` (reason / winner / token+turn totals / finish time) β and ignores the
|
| 7 |
+
rest. A run that has started but not finished yields a summary with the ``finished_*``
|
| 8 |
+
fields left ``None`` / zero.
|
| 9 |
+
|
| 10 |
+
Two implementations, one contract:
|
| 11 |
+
|
| 12 |
+
* :func:`index_runs` is the reference oracle β a pure function over any
|
| 13 |
+
``Iterable[Event]``, folding per ``run_id`` in first-seen order. Tests and offline
|
| 14 |
+
callers use this.
|
| 15 |
+
* :func:`index_runs_from_ledger` produces the *same* list straight from a ledger's
|
| 16 |
+
indexed queries (``runs()`` + ``events_for_run()``), so a hosted SQL backend never
|
| 17 |
+
has to stream every event through Python to list its runs.
|
| 18 |
+
|
| 19 |
+
The cast payload mirrors the enriched ``run.started`` shape (see ``src/core/events.py``):
|
| 20 |
+
``{agent_name: {"model_endpoint": str | None, "model_profile": str | None}}``.
|
| 21 |
+
"""
|
| 22 |
+
|
| 23 |
+
from __future__ import annotations
|
| 24 |
+
|
| 25 |
+
from datetime import datetime
|
| 26 |
+
from typing import TYPE_CHECKING, Any, Iterable
|
| 27 |
+
|
| 28 |
+
from pydantic import BaseModel, ConfigDict, Field
|
| 29 |
+
|
| 30 |
+
from src.core.events import Event
|
| 31 |
+
|
| 32 |
+
if TYPE_CHECKING: # avoid a hard import cycle / SQLAlchemy dependency on the offline path
|
| 33 |
+
from src.core.ledger import Ledger
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
class CastBinding(BaseModel):
|
| 37 |
+
"""The model a cast member is bound to (mirrors the ``run.started`` cast entry)."""
|
| 38 |
+
|
| 39 |
+
model_config = ConfigDict(extra="ignore")
|
| 40 |
+
|
| 41 |
+
model_endpoint: str | None = None
|
| 42 |
+
model_profile: str | None = None
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
class RunSummary(BaseModel):
|
| 46 |
+
"""A one-line summary of a single run, folded from its bookend events."""
|
| 47 |
+
|
| 48 |
+
model_config = ConfigDict(extra="forbid")
|
| 49 |
+
|
| 50 |
+
run_id: str
|
| 51 |
+
scenario: str = ""
|
| 52 |
+
seed: str = ""
|
| 53 |
+
session_id: str | None = None
|
| 54 |
+
cast: dict[str, CastBinding] = Field(default_factory=dict)
|
| 55 |
+
started_at: datetime | None = None
|
| 56 |
+
finished_at: datetime | None = None
|
| 57 |
+
reason: str | None = None
|
| 58 |
+
winner: str | None = None
|
| 59 |
+
winning_model: str | None = None
|
| 60 |
+
turns: int = 0
|
| 61 |
+
tokens: int = 0
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def _coerce_cast(raw: Any) -> dict[str, CastBinding]:
|
| 65 |
+
if not isinstance(raw, dict):
|
| 66 |
+
return {}
|
| 67 |
+
cast: dict[str, CastBinding] = {}
|
| 68 |
+
for name, binding in raw.items():
|
| 69 |
+
if isinstance(binding, dict):
|
| 70 |
+
cast[name] = CastBinding(**binding)
|
| 71 |
+
else:
|
| 72 |
+
cast[name] = CastBinding()
|
| 73 |
+
return cast
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def _apply_started(summary: RunSummary, event: Event) -> None:
|
| 77 |
+
payload = event.payload
|
| 78 |
+
summary.scenario = payload.get("scenario", "") or ""
|
| 79 |
+
summary.seed = payload.get("seed", "") or ""
|
| 80 |
+
# Attribution lives on the envelope (stamped by the conductor on every event);
|
| 81 |
+
# the run.started payload copy is kept for human-readable traces.
|
| 82 |
+
summary.session_id = event.session_id or payload.get("session_id") or None
|
| 83 |
+
summary.cast = _coerce_cast(payload.get("cast"))
|
| 84 |
+
summary.started_at = event.created_at
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def _apply_finished(summary: RunSummary, event: Event) -> None:
|
| 88 |
+
payload = event.payload
|
| 89 |
+
summary.reason = payload.get("reason")
|
| 90 |
+
summary.winner = payload.get("winner")
|
| 91 |
+
summary.winning_model = payload.get("winning_model")
|
| 92 |
+
summary.turns = int(payload.get("turns", 0) or 0)
|
| 93 |
+
summary.tokens = int(payload.get("tokens", 0) or 0)
|
| 94 |
+
summary.finished_at = event.created_at
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def index_runs(events: Iterable[Event]) -> list[RunSummary]:
|
| 98 |
+
"""Fold *events* into one :class:`RunSummary` per run, in first-seen order.
|
| 99 |
+
|
| 100 |
+
Pure reference oracle: ``run.started`` populates scenario/seed/cast/started_at and
|
| 101 |
+
``run.finished`` populates reason/winner/winning_model/turns/tokens/finished_at.
|
| 102 |
+
Runs appear in the order their *first* event is seen. All other event kinds are
|
| 103 |
+
ignored (they don't change the summary), but any event is enough to register a run.
|
| 104 |
+
"""
|
| 105 |
+
summaries: dict[str, RunSummary] = {}
|
| 106 |
+
for event in events:
|
| 107 |
+
summary = summaries.get(event.run_id)
|
| 108 |
+
if summary is None:
|
| 109 |
+
summary = RunSummary(run_id=event.run_id)
|
| 110 |
+
summaries[event.run_id] = summary
|
| 111 |
+
if event.kind == "run.started":
|
| 112 |
+
_apply_started(summary, event)
|
| 113 |
+
elif event.kind == "run.finished":
|
| 114 |
+
_apply_finished(summary, event)
|
| 115 |
+
return list(summaries.values())
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def index_runs_from_ledger(ledger: "Ledger") -> list[RunSummary]:
|
| 119 |
+
"""Build the same :class:`RunSummary` list using a ledger's indexed queries.
|
| 120 |
+
|
| 121 |
+
Equivalent to ``index_runs(ledger.events)`` but reads run-scoped slices via
|
| 122 |
+
``ledger.runs()`` + ``ledger.events_for_run()`` so SQL backends keep the work in the
|
| 123 |
+
database's indexes rather than streaming the whole log through Python. ``index_runs``
|
| 124 |
+
remains the oracle the two paths are checked against.
|
| 125 |
+
"""
|
| 126 |
+
summaries: list[RunSummary] = []
|
| 127 |
+
for run_id in ledger.runs():
|
| 128 |
+
summary = RunSummary(run_id=run_id)
|
| 129 |
+
for event in ledger.events_for_run(run_id):
|
| 130 |
+
if event.kind == "run.started":
|
| 131 |
+
_apply_started(summary, event)
|
| 132 |
+
elif event.kind == "run.finished":
|
| 133 |
+
_apply_finished(summary, event)
|
| 134 |
+
summaries.append(summary)
|
| 135 |
+
return summaries
|
|
@@ -26,6 +26,7 @@ installed. The offline in-memory path stays import-clean.
|
|
| 26 |
Backend selection lives in :mod:`src.core.ledger_factory`: with ``DATABASE_URL``
|
| 27 |
set the system uses this store; otherwise it uses the in-memory ``Ledger``.
|
| 28 |
"""
|
|
|
|
| 29 |
from __future__ import annotations
|
| 30 |
|
| 31 |
import json
|
|
@@ -95,6 +96,7 @@ class SqlAlchemyLedger(Ledger):
|
|
| 95 |
Column("payload", Text, nullable=False),
|
| 96 |
Column("created_at", DateTime(timezone=True), nullable=False),
|
| 97 |
Column("schema_version", Integer, nullable=False, server_default="1"),
|
|
|
|
| 98 |
)
|
| 99 |
self._metadata.create_all(self._engine)
|
| 100 |
|
|
@@ -121,6 +123,7 @@ class SqlAlchemyLedger(Ledger):
|
|
| 121 |
payload=json.dumps(event.payload),
|
| 122 |
created_at=_aware(event.created_at),
|
| 123 |
schema_version=event.schema_version,
|
|
|
|
| 124 |
)
|
| 125 |
)
|
| 126 |
self._cache.append(event)
|
|
@@ -179,6 +182,26 @@ class SqlAlchemyLedger(Ledger):
|
|
| 179 |
value = conn.execute(stmt).scalar()
|
| 180 |
return value or 0
|
| 181 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
def close(self) -> None:
|
| 183 |
self._engine.dispose()
|
| 184 |
|
|
@@ -219,6 +242,7 @@ class SqlAlchemyLedger(Ledger):
|
|
| 219 |
payload=payload,
|
| 220 |
created_at=created_at,
|
| 221 |
schema_version=row["schema_version"],
|
|
|
|
| 222 |
)
|
| 223 |
|
| 224 |
|
|
|
|
| 26 |
Backend selection lives in :mod:`src.core.ledger_factory`: with ``DATABASE_URL``
|
| 27 |
set the system uses this store; otherwise it uses the in-memory ``Ledger``.
|
| 28 |
"""
|
| 29 |
+
|
| 30 |
from __future__ import annotations
|
| 31 |
|
| 32 |
import json
|
|
|
|
| 96 |
Column("payload", Text, nullable=False),
|
| 97 |
Column("created_at", DateTime(timezone=True), nullable=False),
|
| 98 |
Column("schema_version", Integer, nullable=False, server_default="1"),
|
| 99 |
+
Column("session_id", String, nullable=True, index=True),
|
| 100 |
)
|
| 101 |
self._metadata.create_all(self._engine)
|
| 102 |
|
|
|
|
| 123 |
payload=json.dumps(event.payload),
|
| 124 |
created_at=_aware(event.created_at),
|
| 125 |
schema_version=event.schema_version,
|
| 126 |
+
session_id=event.session_id,
|
| 127 |
)
|
| 128 |
)
|
| 129 |
self._cache.append(event)
|
|
|
|
| 182 |
value = conn.execute(stmt).scalar()
|
| 183 |
return value or 0
|
| 184 |
|
| 185 |
+
def events_for_run(self, run_id: str) -> tuple[Event, ...]:
|
| 186 |
+
"""Return the events of *run_id* in append/offset order (indexed query)."""
|
| 187 |
+
from sqlalchemy import select
|
| 188 |
+
|
| 189 |
+
t = self._events_table
|
| 190 |
+
stmt = select(t).where(t.c.run_id == run_id).order_by(t.c.offset)
|
| 191 |
+
with self._engine.connect() as conn:
|
| 192 |
+
rows = conn.execute(stmt).mappings().all()
|
| 193 |
+
return tuple(self._row_to_event(row) for row in rows)
|
| 194 |
+
|
| 195 |
+
def runs(self) -> tuple[str, ...]:
|
| 196 |
+
"""Return the distinct run_ids in first-seen order (indexed query)."""
|
| 197 |
+
from sqlalchemy import func, select
|
| 198 |
+
|
| 199 |
+
t = self._events_table
|
| 200 |
+
stmt = select(t.c.run_id).group_by(t.c.run_id).order_by(func.min(t.c.offset))
|
| 201 |
+
with self._engine.connect() as conn:
|
| 202 |
+
rows = conn.execute(stmt).scalars().all()
|
| 203 |
+
return tuple(rows)
|
| 204 |
+
|
| 205 |
def close(self) -> None:
|
| 206 |
self._engine.dispose()
|
| 207 |
|
|
|
|
| 242 |
payload=payload,
|
| 243 |
created_at=created_at,
|
| 244 |
schema_version=row["schema_version"],
|
| 245 |
+
session_id=row.get("session_id"),
|
| 246 |
)
|
| 247 |
|
| 248 |
|
|
@@ -13,6 +13,7 @@ Design decisions:
|
|
| 13 |
- reset() is deliberately destructive: it clears the current run, not all runs.
|
| 14 |
Multi-run persistence (keeping history across resets) is a Phase 3 milestone.
|
| 15 |
"""
|
|
|
|
| 16 |
from __future__ import annotations
|
| 17 |
|
| 18 |
import json
|
|
@@ -55,11 +56,13 @@ class SQLiteLedger(Ledger):
|
|
| 55 |
actor TEXT NOT NULL,
|
| 56 |
payload TEXT NOT NULL,
|
| 57 |
created_at TEXT NOT NULL,
|
| 58 |
-
schema_version INTEGER NOT NULL DEFAULT 1
|
|
|
|
| 59 |
);
|
| 60 |
CREATE INDEX IF NOT EXISTS idx_run_id ON events(run_id);
|
| 61 |
CREATE INDEX IF NOT EXISTS idx_kind ON events(kind);
|
| 62 |
CREATE INDEX IF NOT EXISTS idx_actor ON events(actor);
|
|
|
|
| 63 |
""")
|
| 64 |
self._conn.commit()
|
| 65 |
|
|
@@ -71,8 +74,8 @@ class SQLiteLedger(Ledger):
|
|
| 71 |
try:
|
| 72 |
self._conn.execute(
|
| 73 |
"INSERT INTO events "
|
| 74 |
-
"(id, run_id, turn, kind, actor, payload, created_at, schema_version) "
|
| 75 |
-
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
|
| 76 |
(
|
| 77 |
event.id,
|
| 78 |
event.run_id,
|
|
@@ -82,6 +85,7 @@ class SQLiteLedger(Ledger):
|
|
| 82 |
json.dumps(event.payload),
|
| 83 |
event.created_at.isoformat(),
|
| 84 |
event.schema_version,
|
|
|
|
| 85 |
),
|
| 86 |
)
|
| 87 |
self._conn.commit()
|
|
@@ -118,52 +122,59 @@ class SQLiteLedger(Ledger):
|
|
| 118 |
ledger = cls(path)
|
| 119 |
return ledger
|
| 120 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
def _load_cache(self) -> None:
|
| 122 |
-
rows = self._conn.execute(
|
| 123 |
-
"SELECT id, run_id, turn, kind, actor, payload, created_at, schema_version "
|
| 124 |
-
"FROM events ORDER BY offset"
|
| 125 |
-
).fetchall()
|
| 126 |
for row in rows:
|
| 127 |
-
|
| 128 |
-
created_at = datetime.fromisoformat(row[6])
|
| 129 |
-
if created_at.tzinfo is None:
|
| 130 |
-
created_at = created_at.replace(tzinfo=timezone.utc)
|
| 131 |
-
except (ValueError, TypeError):
|
| 132 |
-
created_at = datetime.now(timezone.utc)
|
| 133 |
-
|
| 134 |
-
e = Event(
|
| 135 |
-
id=row[0],
|
| 136 |
-
run_id=row[1],
|
| 137 |
-
turn=row[2],
|
| 138 |
-
kind=row[3], # type: ignore[arg-type]
|
| 139 |
-
actor=row[4],
|
| 140 |
-
payload=json.loads(row[5]),
|
| 141 |
-
created_at=created_at,
|
| 142 |
-
schema_version=row[7],
|
| 143 |
-
)
|
| 144 |
self._cache.append(e)
|
| 145 |
self._seen_ids.add(e.id)
|
| 146 |
|
| 147 |
def tail(self, from_offset: int = 0) -> tuple[Event, ...]:
|
| 148 |
"""Return events with offset > from_offset (for crash-recovery replay)."""
|
| 149 |
rows = self._conn.execute(
|
| 150 |
-
"SELECT
|
| 151 |
-
"FROM events WHERE offset > ? ORDER BY offset",
|
| 152 |
(from_offset,),
|
| 153 |
).fetchall()
|
| 154 |
-
|
| 155 |
-
for row in rows:
|
| 156 |
-
e = Event(
|
| 157 |
-
id=row[0], run_id=row[1], turn=row[2], kind=row[3], # type: ignore[arg-type]
|
| 158 |
-
actor=row[4], payload=json.loads(row[5]),
|
| 159 |
-
schema_version=row[7],
|
| 160 |
-
)
|
| 161 |
-
events.append(e)
|
| 162 |
-
return tuple(events)
|
| 163 |
|
| 164 |
def latest_offset(self) -> int:
|
| 165 |
row = self._conn.execute("SELECT MAX(offset) FROM events").fetchone()
|
| 166 |
return row[0] or 0
|
| 167 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 168 |
def close(self) -> None:
|
| 169 |
self._conn.close()
|
|
|
|
| 13 |
- reset() is deliberately destructive: it clears the current run, not all runs.
|
| 14 |
Multi-run persistence (keeping history across resets) is a Phase 3 milestone.
|
| 15 |
"""
|
| 16 |
+
|
| 17 |
from __future__ import annotations
|
| 18 |
|
| 19 |
import json
|
|
|
|
| 56 |
actor TEXT NOT NULL,
|
| 57 |
payload TEXT NOT NULL,
|
| 58 |
created_at TEXT NOT NULL,
|
| 59 |
+
schema_version INTEGER NOT NULL DEFAULT 1,
|
| 60 |
+
session_id TEXT
|
| 61 |
);
|
| 62 |
CREATE INDEX IF NOT EXISTS idx_run_id ON events(run_id);
|
| 63 |
CREATE INDEX IF NOT EXISTS idx_kind ON events(kind);
|
| 64 |
CREATE INDEX IF NOT EXISTS idx_actor ON events(actor);
|
| 65 |
+
CREATE INDEX IF NOT EXISTS idx_session_id ON events(session_id);
|
| 66 |
""")
|
| 67 |
self._conn.commit()
|
| 68 |
|
|
|
|
| 74 |
try:
|
| 75 |
self._conn.execute(
|
| 76 |
"INSERT INTO events "
|
| 77 |
+
"(id, run_id, turn, kind, actor, payload, created_at, schema_version, session_id) "
|
| 78 |
+
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
|
| 79 |
(
|
| 80 |
event.id,
|
| 81 |
event.run_id,
|
|
|
|
| 85 |
json.dumps(event.payload),
|
| 86 |
event.created_at.isoformat(),
|
| 87 |
event.schema_version,
|
| 88 |
+
event.session_id,
|
| 89 |
),
|
| 90 |
)
|
| 91 |
self._conn.commit()
|
|
|
|
| 122 |
ledger = cls(path)
|
| 123 |
return ledger
|
| 124 |
|
| 125 |
+
_SELECT_COLS = "id, run_id, turn, kind, actor, payload, created_at, schema_version, session_id"
|
| 126 |
+
|
| 127 |
+
@staticmethod
|
| 128 |
+
def _row_to_event(row: tuple) -> Event:
|
| 129 |
+
try:
|
| 130 |
+
created_at = datetime.fromisoformat(row[6])
|
| 131 |
+
if created_at.tzinfo is None:
|
| 132 |
+
created_at = created_at.replace(tzinfo=timezone.utc)
|
| 133 |
+
except (ValueError, TypeError):
|
| 134 |
+
created_at = datetime.now(timezone.utc)
|
| 135 |
+
return Event(
|
| 136 |
+
id=row[0],
|
| 137 |
+
run_id=row[1],
|
| 138 |
+
turn=row[2],
|
| 139 |
+
kind=row[3], # type: ignore[arg-type]
|
| 140 |
+
actor=row[4],
|
| 141 |
+
payload=json.loads(row[5]),
|
| 142 |
+
created_at=created_at,
|
| 143 |
+
schema_version=row[7],
|
| 144 |
+
session_id=row[8],
|
| 145 |
+
)
|
| 146 |
+
|
| 147 |
def _load_cache(self) -> None:
|
| 148 |
+
rows = self._conn.execute(f"SELECT {self._SELECT_COLS} FROM events ORDER BY offset").fetchall()
|
|
|
|
|
|
|
|
|
|
| 149 |
for row in rows:
|
| 150 |
+
e = self._row_to_event(row)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
self._cache.append(e)
|
| 152 |
self._seen_ids.add(e.id)
|
| 153 |
|
| 154 |
def tail(self, from_offset: int = 0) -> tuple[Event, ...]:
|
| 155 |
"""Return events with offset > from_offset (for crash-recovery replay)."""
|
| 156 |
rows = self._conn.execute(
|
| 157 |
+
f"SELECT {self._SELECT_COLS} FROM events WHERE offset > ? ORDER BY offset",
|
|
|
|
| 158 |
(from_offset,),
|
| 159 |
).fetchall()
|
| 160 |
+
return tuple(self._row_to_event(row) for row in rows)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 161 |
|
| 162 |
def latest_offset(self) -> int:
|
| 163 |
row = self._conn.execute("SELECT MAX(offset) FROM events").fetchone()
|
| 164 |
return row[0] or 0
|
| 165 |
|
| 166 |
+
def events_for_run(self, run_id: str) -> tuple[Event, ...]:
|
| 167 |
+
"""Return the events of *run_id* in append/offset order (indexed query)."""
|
| 168 |
+
rows = self._conn.execute(
|
| 169 |
+
f"SELECT {self._SELECT_COLS} FROM events WHERE run_id = ? ORDER BY offset",
|
| 170 |
+
(run_id,),
|
| 171 |
+
).fetchall()
|
| 172 |
+
return tuple(self._row_to_event(row) for row in rows)
|
| 173 |
+
|
| 174 |
+
def runs(self) -> tuple[str, ...]:
|
| 175 |
+
"""Return the distinct run_ids in first-seen order (indexed query)."""
|
| 176 |
+
rows = self._conn.execute("SELECT run_id FROM events GROUP BY run_id ORDER BY MIN(offset)").fetchall()
|
| 177 |
+
return tuple(row[0] for row in rows)
|
| 178 |
+
|
| 179 |
def close(self) -> None:
|
| 180 |
self._conn.close()
|
|
@@ -30,6 +30,20 @@ from src.tools.builtins import default_tool_registry
|
|
| 30 |
from src.ui.fishbowl.adapter import scenario_voice
|
| 31 |
from src.ui.fishbowl.view_model import view_model_at
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
# ββ loop-safety backstop ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 34 |
# Belt-and-suspenders against a runaway autoplay loop: even when the governor never
|
| 35 |
# trips (e.g. a generous budget) the timer halts after this many consecutive auto-ticks
|
|
@@ -384,6 +398,12 @@ def advance_one_tick(session: FishbowlSession | None, k: int, ticks: int, *, max
|
|
| 384 |
ticks = int(ticks or 0)
|
| 385 |
if session is None:
|
| 386 |
return 0, 0, None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 387 |
if session.has_verdict():
|
| 388 |
return k, ticks, "verdict reached β the show resolved"
|
| 389 |
if k < session.head:
|
|
@@ -479,10 +499,33 @@ def build_app() -> gr.Blocks:
|
|
| 479 |
blank_state = gr.State("") # stand-in input when a leaf widget is absent
|
| 480 |
stopped_state = gr.State(False) # set once the run halts (budget/backstop/verdict)
|
| 481 |
tick_count_state = gr.State(0) # consecutive autoplay ticks (the 40-tick backstop)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 482 |
|
| 483 |
with gr.Tabs() as tabs:
|
| 484 |
with gr.Tab("The Lab", id="lab"):
|
| 485 |
lab_handles = build_lab()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 486 |
with gr.Tab("The Show", id="show"):
|
| 487 |
show_handles = build_show()
|
| 488 |
with gr.Tab("Telemetry", id="telemetry"):
|
|
@@ -491,6 +534,10 @@ def build_app() -> gr.Blocks:
|
|
| 491 |
# CRT foreground layers (scanlines + vignette, above content, click-through).
|
| 492 |
gr.HTML(_CRT_FG_HTML)
|
| 493 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 494 |
_wire(
|
| 495 |
tabs=tabs,
|
| 496 |
lab_handles=lab_handles or {},
|
|
@@ -503,8 +550,13 @@ def build_app() -> gr.Blocks:
|
|
| 503 |
blank_state=blank_state,
|
| 504 |
stopped_state=stopped_state,
|
| 505 |
tick_count_state=tick_count_state,
|
|
|
|
| 506 |
)
|
| 507 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 508 |
return demo
|
| 509 |
|
| 510 |
|
|
@@ -532,6 +584,7 @@ def _wire(
|
|
| 532 |
blank_state: gr.State,
|
| 533 |
stopped_state: gr.State,
|
| 534 |
tick_count_state: gr.State,
|
|
|
|
| 535 |
) -> None:
|
| 536 |
"""Connect Lab/Show component handles to session transport + HTML re-render.
|
| 537 |
|
|
@@ -688,6 +741,7 @@ def _wire(
|
|
| 688 |
hourly_budget_usd,
|
| 689 |
layout,
|
| 690 |
mind_reader,
|
|
|
|
| 691 |
):
|
| 692 |
title = _scenario_title(scenario_value)
|
| 693 |
name = SCENARIOS.get(title, "")
|
|
@@ -716,7 +770,7 @@ def _wire(
|
|
| 716 |
else None
|
| 717 |
)
|
| 718 |
if session is not None:
|
| 719 |
-
session.reset((seed_value or "").strip())
|
| 720 |
k = session.head
|
| 721 |
else:
|
| 722 |
k = 0
|
|
@@ -746,6 +800,7 @@ def _wire(
|
|
| 746 |
hourly_budget_in if hourly_budget_in is not None else blank_state,
|
| 747 |
layout_state,
|
| 748 |
mind_reader_state,
|
|
|
|
| 749 |
]
|
| 750 |
summon_btn.click(
|
| 751 |
on_summon,
|
|
@@ -962,6 +1017,123 @@ def _pad_values(values, show_outs: list) -> tuple:
|
|
| 962 |
return tuple(values[: len(show_outs)])
|
| 963 |
|
| 964 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 965 |
# ββ dev server port (ported from the original root app.py) ββββββββββββββββββββββ
|
| 966 |
|
| 967 |
|
|
|
|
| 30 |
from src.ui.fishbowl.adapter import scenario_voice
|
| 31 |
from src.ui.fishbowl.view_model import view_model_at
|
| 32 |
|
| 33 |
+
try: # archive: "my past sessions" read layer over the run-history ledger (ADR-0026)
|
| 34 |
+
from src.ui.fishbowl.archive import list_runs, load_replay, run_card_label
|
| 35 |
+
except Exception: # pragma: no cover - degrade gracefully if the archive unit is absent
|
| 36 |
+
|
| 37 |
+
def list_runs(*_args, **_kwargs):
|
| 38 |
+
return []
|
| 39 |
+
|
| 40 |
+
def load_replay(*_args, **_kwargs):
|
| 41 |
+
return None
|
| 42 |
+
|
| 43 |
+
def run_card_label(_summary):
|
| 44 |
+
return "βΆ (run)"
|
| 45 |
+
|
| 46 |
+
|
| 47 |
# ββ loop-safety backstop ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 48 |
# Belt-and-suspenders against a runaway autoplay loop: even when the governor never
|
| 49 |
# trips (e.g. a generous budget) the timer halts after this many consecutive auto-ticks
|
|
|
|
| 398 |
ticks = int(ticks or 0)
|
| 399 |
if session is None:
|
| 400 |
return 0, 0, None
|
| 401 |
+
if getattr(session, "replay", False):
|
| 402 |
+
# A loaded past run: replay forward through the recorded prefix, then stop.
|
| 403 |
+
# It owns no live engine, so generating is impossible β never spend a tick here.
|
| 404 |
+
if k < session.head:
|
| 405 |
+
return k + 1, ticks, None
|
| 406 |
+
return k, ticks, "end of session β replay complete"
|
| 407 |
if session.has_verdict():
|
| 408 |
return k, ticks, "verdict reached β the show resolved"
|
| 409 |
if k < session.head:
|
|
|
|
| 499 |
blank_state = gr.State("") # stand-in input when a leaf widget is absent
|
| 500 |
stopped_state = gr.State(False) # set once the run halts (budget/backstop/verdict)
|
| 501 |
tick_count_state = gr.State(0) # consecutive autoplay ticks (the 40-tick backstop)
|
| 502 |
+
# Per-user session id: resolved from the browser's localStorage on load (see
|
| 503 |
+
# _SESSION_ID_JS) so it survives reloads. Stamped onto run.started so the
|
| 504 |
+
# Archive can list "my sessions only"; a hidden carrier, never shown.
|
| 505 |
+
session_id_box = gr.Textbox(value="", visible=False, elem_id="fb-session-id")
|
| 506 |
+
|
| 507 |
+
# Populated after the tabs build, then read at gr.render runtime (post-load) so
|
| 508 |
+
# the Lab's Archive drawer can target the Show's panes it lists into.
|
| 509 |
+
archive_refs: dict = {}
|
| 510 |
|
| 511 |
with gr.Tabs() as tabs:
|
| 512 |
with gr.Tab("The Lab", id="lab"):
|
| 513 |
lab_handles = build_lab()
|
| 514 |
+
_build_archive_drawer(
|
| 515 |
+
scenario_handle=(lab_handles or {}).get("scenario"),
|
| 516 |
+
session_id_box=session_id_box,
|
| 517 |
+
refs=archive_refs,
|
| 518 |
+
tabs=tabs,
|
| 519 |
+
states={
|
| 520 |
+
"session": session_state,
|
| 521 |
+
"k": k_state,
|
| 522 |
+
"scenario": scenario_state,
|
| 523 |
+
"layout": layout_state,
|
| 524 |
+
"mind": mind_reader_state,
|
| 525 |
+
"stopped": stopped_state,
|
| 526 |
+
"ticks": tick_count_state,
|
| 527 |
+
},
|
| 528 |
+
)
|
| 529 |
with gr.Tab("The Show", id="show"):
|
| 530 |
show_handles = build_show()
|
| 531 |
with gr.Tab("Telemetry", id="telemetry"):
|
|
|
|
| 534 |
# CRT foreground layers (scanlines + vignette, above content, click-through).
|
| 535 |
gr.HTML(_CRT_FG_HTML)
|
| 536 |
|
| 537 |
+
# The Archive's gr.render runs client-side after build; by then show_handles
|
| 538 |
+
# exists, so it reads the live panes through this ref.
|
| 539 |
+
archive_refs["show_handles"] = show_handles or {}
|
| 540 |
+
|
| 541 |
_wire(
|
| 542 |
tabs=tabs,
|
| 543 |
lab_handles=lab_handles or {},
|
|
|
|
| 550 |
blank_state=blank_state,
|
| 551 |
stopped_state=stopped_state,
|
| 552 |
tick_count_state=tick_count_state,
|
| 553 |
+
session_id_box=session_id_box,
|
| 554 |
)
|
| 555 |
|
| 556 |
+
# Resolve (or mint) the browser's session id once the page loads; updating the
|
| 557 |
+
# hidden box fires its change, which re-renders the Archive list for this user.
|
| 558 |
+
demo.load(None, None, [session_id_box], js=_SESSION_ID_JS)
|
| 559 |
+
|
| 560 |
return demo
|
| 561 |
|
| 562 |
|
|
|
|
| 584 |
blank_state: gr.State,
|
| 585 |
stopped_state: gr.State,
|
| 586 |
tick_count_state: gr.State,
|
| 587 |
+
session_id_box: gr.Textbox | None = None,
|
| 588 |
) -> None:
|
| 589 |
"""Connect Lab/Show component handles to session transport + HTML re-render.
|
| 590 |
|
|
|
|
| 741 |
hourly_budget_usd,
|
| 742 |
layout,
|
| 743 |
mind_reader,
|
| 744 |
+
session_id,
|
| 745 |
):
|
| 746 |
title = _scenario_title(scenario_value)
|
| 747 |
name = SCENARIOS.get(title, "")
|
|
|
|
| 770 |
else None
|
| 771 |
)
|
| 772 |
if session is not None:
|
| 773 |
+
session.reset((seed_value or "").strip(), session_id=(session_id or "").strip() or None)
|
| 774 |
k = session.head
|
| 775 |
else:
|
| 776 |
k = 0
|
|
|
|
| 800 |
hourly_budget_in if hourly_budget_in is not None else blank_state,
|
| 801 |
layout_state,
|
| 802 |
mind_reader_state,
|
| 803 |
+
session_id_box if session_id_box is not None else blank_state,
|
| 804 |
]
|
| 805 |
summon_btn.click(
|
| 806 |
on_summon,
|
|
|
|
| 1017 |
return tuple(values[: len(show_outs)])
|
| 1018 |
|
| 1019 |
|
| 1020 |
+
# ββ Archive drawer ("my past sessions" β read-only replay) ββββββββββββββββββββββ
|
| 1021 |
+
|
| 1022 |
+
# Resolve (or mint) a per-browser session id from localStorage on page load. Kept in
|
| 1023 |
+
# JS so the id is the user's own and survives reloads β Python only reads the carrier.
|
| 1024 |
+
_SESSION_ID_JS = """
|
| 1025 |
+
() => {
|
| 1026 |
+
try {
|
| 1027 |
+
let id = localStorage.getItem('fishbowl_session_id');
|
| 1028 |
+
if (!id) {
|
| 1029 |
+
id = (window.crypto && crypto.randomUUID)
|
| 1030 |
+
? crypto.randomUUID()
|
| 1031 |
+
: 'sess-' + Math.random().toString(36).slice(2) + Date.now().toString(36);
|
| 1032 |
+
localStorage.setItem('fishbowl_session_id', id);
|
| 1033 |
+
}
|
| 1034 |
+
return id;
|
| 1035 |
+
} catch (e) {
|
| 1036 |
+
return 'sess-' + Math.random().toString(36).slice(2);
|
| 1037 |
+
}
|
| 1038 |
+
}
|
| 1039 |
+
"""
|
| 1040 |
+
|
| 1041 |
+
|
| 1042 |
+
def _title_for(value) -> str:
|
| 1043 |
+
"""Resolve a scenario title *or* internal name to a display-title key."""
|
| 1044 |
+
if value in SCENARIOS:
|
| 1045 |
+
return value
|
| 1046 |
+
for title, name in SCENARIOS.items():
|
| 1047 |
+
if value == name:
|
| 1048 |
+
return title
|
| 1049 |
+
return _DEFAULT_TITLE
|
| 1050 |
+
|
| 1051 |
+
|
| 1052 |
+
def _show_outs(show_handles: dict) -> list:
|
| 1053 |
+
"""The present Show panes (stage, feed, meters, verdict), in render order."""
|
| 1054 |
+
stage = _h(show_handles, "stage", "stage_html", "constellation")
|
| 1055 |
+
feed = _h(show_handles, "feed", "feed_html")
|
| 1056 |
+
meters = _h(show_handles, "meters", "meters_html")
|
| 1057 |
+
verdict = _h(show_handles, "verdict", "verdict_html")
|
| 1058 |
+
return [c for c in (stage, feed, meters, verdict) if c is not None]
|
| 1059 |
+
|
| 1060 |
+
|
| 1061 |
+
def _archive_empty_html() -> str:
|
| 1062 |
+
"""The drawer's empty state β no past runs for this world yet."""
|
| 1063 |
+
return _fishbowl(
|
| 1064 |
+
'<div class="archive-empty">'
|
| 1065 |
+
'<div class="eyebrow">⟳ No past sessions</div>'
|
| 1066 |
+
'<div class="ae-body">Summon the bowl, and your runs in this world '
|
| 1067 |
+
"will gather here β yours alone, replayable any time.</div>"
|
| 1068 |
+
"</div>",
|
| 1069 |
+
role="fb-archive",
|
| 1070 |
+
)
|
| 1071 |
+
|
| 1072 |
+
|
| 1073 |
+
def _build_archive_drawer(*, scenario_handle, session_id_box, refs: dict, tabs, states: dict) -> None:
|
| 1074 |
+
"""The Lab's "Past sessions" accordion: clickable phosphor cards β read-only replay.
|
| 1075 |
+
|
| 1076 |
+
A ``gr.render`` keyed on (scenario, session id) lists *this user's* runs for the
|
| 1077 |
+
*current* world via :func:`list_runs`; clicking a card loads that run with
|
| 1078 |
+
:func:`load_replay` and jumps to the Show. The list re-renders on world change,
|
| 1079 |
+
when the session id resolves from localStorage, and on the manual refresh.
|
| 1080 |
+
"""
|
| 1081 |
+
if scenario_handle is None: # no scenario picker β nothing to scope a list to
|
| 1082 |
+
return
|
| 1083 |
+
|
| 1084 |
+
with gr.Accordion("β² Past sessions Β· this world", open=False, elem_classes=["archive-drawer"]):
|
| 1085 |
+
refresh = gr.Button("β³ refresh", size="sm", elem_classes=["archive-refresh"], scale=0)
|
| 1086 |
+
|
| 1087 |
+
@gr.render(
|
| 1088 |
+
inputs=[scenario_handle, session_id_box],
|
| 1089 |
+
triggers=[scenario_handle.change, session_id_box.change, refresh.click],
|
| 1090 |
+
)
|
| 1091 |
+
def _render_archive(scenario_value, session_id):
|
| 1092 |
+
name = SCENARIOS.get(_title_for(scenario_value), "")
|
| 1093 |
+
runs = list_runs(name, session_id)
|
| 1094 |
+
if not runs:
|
| 1095 |
+
gr.HTML(_archive_empty_html())
|
| 1096 |
+
return
|
| 1097 |
+
|
| 1098 |
+
show_outs = _show_outs(refs.get("show_handles") or {})
|
| 1099 |
+
n_out = 6 + len(show_outs) # session, k, scenario, tabs, *panes, stopped, ticks
|
| 1100 |
+
|
| 1101 |
+
def _loader(run_id: str):
|
| 1102 |
+
def _load(layout, mind_reader):
|
| 1103 |
+
session = load_replay(run_id, registry=_registry, tools=_tools)
|
| 1104 |
+
if session is None:
|
| 1105 |
+
return tuple(gr.update() for _ in range(n_out))
|
| 1106 |
+
k = session.head # land on the full discussion; scrub/βΆ replays it
|
| 1107 |
+
out = _render_at(session, k, layout=layout, mind_reader=mind_reader)
|
| 1108 |
+
return (
|
| 1109 |
+
session,
|
| 1110 |
+
k,
|
| 1111 |
+
_title_for(session.scenario_name),
|
| 1112 |
+
gr.update(selected="show"),
|
| 1113 |
+
*_pad_values(out, show_outs),
|
| 1114 |
+
False,
|
| 1115 |
+
0,
|
| 1116 |
+
)
|
| 1117 |
+
|
| 1118 |
+
return _load
|
| 1119 |
+
|
| 1120 |
+
for summary in runs:
|
| 1121 |
+
card = gr.Button(run_card_label(summary), elem_classes=["archive-card"])
|
| 1122 |
+
card.click(
|
| 1123 |
+
_loader(summary.run_id),
|
| 1124 |
+
inputs=[states["layout"], states["mind"]],
|
| 1125 |
+
outputs=[
|
| 1126 |
+
states["session"],
|
| 1127 |
+
states["k"],
|
| 1128 |
+
states["scenario"],
|
| 1129 |
+
tabs,
|
| 1130 |
+
*show_outs,
|
| 1131 |
+
states["stopped"],
|
| 1132 |
+
states["ticks"],
|
| 1133 |
+
],
|
| 1134 |
+
)
|
| 1135 |
+
|
| 1136 |
+
|
| 1137 |
# ββ dev server port (ported from the original root app.py) ββββββββββββββββββββββ
|
| 1138 |
|
| 1139 |
|
|
@@ -0,0 +1,145 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""The Archive β "my past sessions" for the Lab's Load drawer.
|
| 2 |
+
|
| 3 |
+
This is a thin read layer over the W1 run-history surface (ADR-0026): it folds the
|
| 4 |
+
shared ledger into per-run :class:`RunSummary` rows via
|
| 5 |
+
:func:`src.core.run_index.index_runs_from_ledger`, then keeps only the runs that
|
| 6 |
+
belong to *this* browser/user (``session_id``) in *this* world (``scenario``). No
|
| 7 |
+
side table is introduced β the run list is a projection of the log, in keeping with
|
| 8 |
+
the single-source-of-truth discipline (ADR-0014).
|
| 9 |
+
|
| 10 |
+
Loading a run hands back a :class:`ReplaySession`: a read-only view the Show scrubs
|
| 11 |
+
without ever generating, so a past session replays verbatim with no token spend.
|
| 12 |
+
"""
|
| 13 |
+
|
| 14 |
+
from __future__ import annotations
|
| 15 |
+
|
| 16 |
+
from src import observability as obs
|
| 17 |
+
from src.core.events import normalize_session_id
|
| 18 |
+
from src.core.ledger_factory import make_ledger
|
| 19 |
+
from src.core.registry import Registry, default_registry
|
| 20 |
+
from src.core.run_index import RunSummary, index_runs_from_ledger
|
| 21 |
+
from src.ui.fishbowl.session import ReplaySession
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def _sort_key(summary: RunSummary):
|
| 25 |
+
# Newest first; runs missing a start time sink to the bottom deterministically.
|
| 26 |
+
started = summary.started_at
|
| 27 |
+
return (started is not None, started)
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def list_runs(
|
| 31 |
+
scenario_name: str,
|
| 32 |
+
session_id: str | None,
|
| 33 |
+
*,
|
| 34 |
+
registry: Registry | None = None,
|
| 35 |
+
limit: int = 50,
|
| 36 |
+
) -> list[RunSummary]:
|
| 37 |
+
"""Past runs for *scenario_name* started by *session_id*, newest first.
|
| 38 |
+
|
| 39 |
+
Returns ``[]`` when the world or the session is unknown β the drawer then shows
|
| 40 |
+
its empty state rather than every user's runs. Scoping to ``session_id`` is what
|
| 41 |
+
makes "my sessions only" hold across browsers (each user has their own id).
|
| 42 |
+
"""
|
| 43 |
+
session_id = normalize_session_id(session_id) # same boundary rule as the write path
|
| 44 |
+
if not scenario_name or not session_id:
|
| 45 |
+
return []
|
| 46 |
+
try:
|
| 47 |
+
ledger = make_ledger()
|
| 48 |
+
except Exception: # pragma: no cover - no event store configured (defensive)
|
| 49 |
+
return []
|
| 50 |
+
summaries = [
|
| 51 |
+
s for s in index_runs_from_ledger(ledger) if s.scenario == scenario_name and s.session_id == session_id
|
| 52 |
+
]
|
| 53 |
+
summaries.sort(key=_sort_key, reverse=True)
|
| 54 |
+
return summaries[:limit]
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def load_replay(
|
| 58 |
+
run_id: str,
|
| 59 |
+
*,
|
| 60 |
+
registry: Registry | None = None,
|
| 61 |
+
tools=None,
|
| 62 |
+
) -> ReplaySession | None:
|
| 63 |
+
"""Build a read-only :class:`ReplaySession` for *run_id*, or ``None`` if unknown.
|
| 64 |
+
|
| 65 |
+
The run names its own scenario on ``run.started`` (ADR-0026), so a replay is
|
| 66 |
+
self-describing β we rebuild the cast/meters from that scenario and replay the
|
| 67 |
+
recorded events verbatim.
|
| 68 |
+
"""
|
| 69 |
+
if not run_id:
|
| 70 |
+
return None
|
| 71 |
+
try:
|
| 72 |
+
ledger = make_ledger()
|
| 73 |
+
except Exception: # pragma: no cover - defensive
|
| 74 |
+
return None
|
| 75 |
+
events = ledger.events_for_run(run_id)
|
| 76 |
+
if not events:
|
| 77 |
+
return None
|
| 78 |
+
started = next((e for e in events if e.kind == "run.started"), None)
|
| 79 |
+
scenario_name = (started.payload.get("scenario") if started is not None else None) or ""
|
| 80 |
+
if not scenario_name:
|
| 81 |
+
return None
|
| 82 |
+
try:
|
| 83 |
+
return ReplaySession(
|
| 84 |
+
run_id=run_id,
|
| 85 |
+
events=events,
|
| 86 |
+
scenario_name=scenario_name,
|
| 87 |
+
registry=registry or default_registry(),
|
| 88 |
+
tools=tools,
|
| 89 |
+
)
|
| 90 |
+
except Exception: # pragma: no cover - scenario no longer in the registry, etc.
|
| 91 |
+
obs.log("archive.load_failed", level="warning", run_id=run_id, scenario=scenario_name)
|
| 92 |
+
return None
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
# ββ card formatting (the phosphor list rows) ββββββββββββββββββββββββββββββββββββββ
|
| 96 |
+
|
| 97 |
+
|
| 98 |
+
def _short_id(run_id: str) -> str:
|
| 99 |
+
"""A stable, glanceable handle for a run β last 4 hex of its uuid."""
|
| 100 |
+
tail = run_id.replace("-", "")[-4:] if run_id else "????"
|
| 101 |
+
return f"#{tail}"
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
def _fmt_tokens(tokens: int) -> str:
|
| 105 |
+
if tokens >= 1000:
|
| 106 |
+
return f"{tokens / 1000:.1f}k tok"
|
| 107 |
+
return f"{tokens} tok"
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
def _fmt_when(summary: RunSummary) -> str:
|
| 111 |
+
started = summary.started_at
|
| 112 |
+
if started is None:
|
| 113 |
+
return "β"
|
| 114 |
+
try:
|
| 115 |
+
return started.strftime("%b %-d Β· %H:%M")
|
| 116 |
+
except ValueError: # pragma: no cover - platforms without %-d
|
| 117 |
+
return started.strftime("%b %d Β· %H:%M")
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
_REASON_LABEL = {
|
| 121 |
+
"verdict": "verdict",
|
| 122 |
+
"budget": "budget",
|
| 123 |
+
"tick_cap": "tick-cap",
|
| 124 |
+
"user_stop": "stopped",
|
| 125 |
+
}
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def run_card_label(summary: RunSummary) -> str:
|
| 129 |
+
"""A compact, single-line label for a run's Load button (styled by CSS).
|
| 130 |
+
|
| 131 |
+
Shape: ``βΆ #a3f9 Β· Jun 12 Β· 14:03 Β· 14t Β· WON Fox Β· verdict``. An unfinished run
|
| 132 |
+
reads ``β¦live`` so the list distinguishes resolved runs from abandoned ones.
|
| 133 |
+
"""
|
| 134 |
+
parts = [f"βΆ {_short_id(summary.run_id)}", _fmt_when(summary)]
|
| 135 |
+
if summary.turns:
|
| 136 |
+
parts.append(f"{summary.turns}t")
|
| 137 |
+
if summary.tokens:
|
| 138 |
+
parts.append(_fmt_tokens(summary.tokens))
|
| 139 |
+
if summary.winner:
|
| 140 |
+
parts.append(f"WON {summary.winner}")
|
| 141 |
+
if summary.reason:
|
| 142 |
+
parts.append(_REASON_LABEL.get(summary.reason, summary.reason))
|
| 143 |
+
else:
|
| 144 |
+
parts.append("β¦live")
|
| 145 |
+
return " Β· ".join(parts)
|
|
@@ -1181,3 +1181,85 @@ footer { display: none !important; }
|
|
| 1181 |
color: var(--ink) !important;
|
| 1182 |
box-shadow: 0 0 14px -4px rgba(79,230,210,0.5) !important;
|
| 1183 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1181 |
color: var(--ink) !important;
|
| 1182 |
box-shadow: 0 0 14px -4px rgba(79,230,210,0.5) !important;
|
| 1183 |
}
|
| 1184 |
+
|
| 1185 |
+
/* ββ Archive drawer: "my past sessions" (Lab β Load β read-only replay) ββββββββββ
|
| 1186 |
+
The accordion + cards are real Gradio components (not inside the `.fishbowl`
|
| 1187 |
+
scope root), so these rules target their `elem_classes` directly and lean on
|
| 1188 |
+
`!important` to win over Gradio's own cascade, like the seed-edit rules above. */
|
| 1189 |
+
.gradio-container .archive-drawer {
|
| 1190 |
+
border: 1px solid var(--line-soft) !important;
|
| 1191 |
+
border-radius: var(--r-lg) !important;
|
| 1192 |
+
background: linear-gradient(180deg, rgba(10,34,43,0.55), rgba(5,18,26,0.35)) !important;
|
| 1193 |
+
margin-top: 4px !important;
|
| 1194 |
+
}
|
| 1195 |
+
.gradio-container .archive-drawer > .label-wrap,
|
| 1196 |
+
.gradio-container .archive-drawer button.label-wrap,
|
| 1197 |
+
.gradio-container .archive-drawer span.label-wrap {
|
| 1198 |
+
font-family: var(--font-display) !important;
|
| 1199 |
+
font-size: 11px !important;
|
| 1200 |
+
letter-spacing: 0.16em !important;
|
| 1201 |
+
text-transform: uppercase !important;
|
| 1202 |
+
color: var(--ink-mid) !important;
|
| 1203 |
+
}
|
| 1204 |
+
|
| 1205 |
+
.gradio-container .archive-refresh {
|
| 1206 |
+
min-width: 0 !important;
|
| 1207 |
+
align-self: flex-end !important;
|
| 1208 |
+
font-family: var(--font-display) !important;
|
| 1209 |
+
font-size: 9.5px !important;
|
| 1210 |
+
letter-spacing: 0.18em !important;
|
| 1211 |
+
text-transform: uppercase !important;
|
| 1212 |
+
color: var(--ink-dim) !important;
|
| 1213 |
+
background: transparent !important;
|
| 1214 |
+
border: 1px solid var(--line-soft) !important;
|
| 1215 |
+
border-radius: var(--r) !important;
|
| 1216 |
+
}
|
| 1217 |
+
.gradio-container .archive-refresh:hover {
|
| 1218 |
+
color: var(--cyan) !important;
|
| 1219 |
+
border-color: var(--line-hot) !important;
|
| 1220 |
+
box-shadow: var(--glow) !important;
|
| 1221 |
+
}
|
| 1222 |
+
|
| 1223 |
+
/* Each past run is a full-width, left-aligned monospace card with a phosphor
|
| 1224 |
+
spine. The whole row is the click target (it loads the replay). */
|
| 1225 |
+
.gradio-container .archive-card {
|
| 1226 |
+
display: block !important;
|
| 1227 |
+
width: 100% !important;
|
| 1228 |
+
text-align: left !important;
|
| 1229 |
+
font-family: var(--font-body) !important;
|
| 1230 |
+
font-size: 12px !important;
|
| 1231 |
+
letter-spacing: 0.02em !important;
|
| 1232 |
+
color: var(--ink) !important;
|
| 1233 |
+
background: var(--panel) !important;
|
| 1234 |
+
border: 1px solid var(--line-soft) !important;
|
| 1235 |
+
border-left: 2px solid var(--teal) !important;
|
| 1236 |
+
border-radius: var(--r) !important;
|
| 1237 |
+
padding: 10px 12px !important;
|
| 1238 |
+
margin: 6px 0 0 !important;
|
| 1239 |
+
white-space: nowrap !important;
|
| 1240 |
+
overflow: hidden !important;
|
| 1241 |
+
text-overflow: ellipsis !important;
|
| 1242 |
+
transition: border-color 0.16s ease, box-shadow 0.16s ease, transform 0.16s ease !important;
|
| 1243 |
+
}
|
| 1244 |
+
.gradio-container .archive-card:hover {
|
| 1245 |
+
border-color: var(--line-hot) !important;
|
| 1246 |
+
border-left-color: var(--cyan) !important;
|
| 1247 |
+
color: var(--cyan) !important;
|
| 1248 |
+
box-shadow: var(--glow) !important;
|
| 1249 |
+
transform: translateX(2px) !important;
|
| 1250 |
+
}
|
| 1251 |
+
|
| 1252 |
+
.fishbowl .archive-empty {
|
| 1253 |
+
border: 1px dashed var(--line-soft);
|
| 1254 |
+
border-radius: var(--r-lg);
|
| 1255 |
+
padding: 18px 16px;
|
| 1256 |
+
background: rgba(5, 18, 26, 0.35);
|
| 1257 |
+
}
|
| 1258 |
+
.fishbowl .archive-empty .ae-body {
|
| 1259 |
+
font-family: var(--font-body);
|
| 1260 |
+
font-size: 12px;
|
| 1261 |
+
color: var(--ink-dim);
|
| 1262 |
+
margin-top: 6px;
|
| 1263 |
+
line-height: 1.5;
|
| 1264 |
+
max-width: 46ch;
|
| 1265 |
+
}
|
|
@@ -59,9 +59,9 @@ class FishbowlSession:
|
|
| 59 |
|
| 60 |
# ββ lifecycle βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 61 |
|
| 62 |
-
def reset(self, seed: str = "") -> None:
|
| 63 |
obs.log("session.reset", scenario=self._scenario_name, seed=seed or self.scenario.default_seed)
|
| 64 |
-
self.conductor.reset(seed or self.scenario.default_seed)
|
| 65 |
|
| 66 |
def step(self, n_ticks: int = 1) -> None:
|
| 67 |
with obs.span("session.step", **{"session.n_ticks": n_ticks}):
|
|
@@ -86,21 +86,50 @@ class FishbowlSession:
|
|
| 86 |
def scenario(self):
|
| 87 |
return self.conductor.scenario
|
| 88 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 89 |
@property
|
| 90 |
def events(self):
|
| 91 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 92 |
|
| 93 |
@property
|
| 94 |
def head(self) -> int:
|
| 95 |
-
"""The generation-head: number of events in
|
| 96 |
-
return len(self.
|
| 97 |
|
| 98 |
def has_verdict(self) -> bool:
|
| 99 |
-
"""True once a ``judge.verdict`` event sits in
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
@property
|
| 106 |
def cast(self) -> list[AgentManifest]:
|
|
@@ -152,3 +181,97 @@ class FishbowlSession:
|
|
| 152 |
token_ceiling=self.token_ceiling,
|
| 153 |
max_rounds=self.max_rounds,
|
| 154 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
# ββ lifecycle βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 61 |
|
| 62 |
+
def reset(self, seed: str = "", *, session_id: str | None = None) -> None:
|
| 63 |
obs.log("session.reset", scenario=self._scenario_name, seed=seed or self.scenario.default_seed)
|
| 64 |
+
self.conductor.reset(seed or self.scenario.default_seed, session_id=session_id)
|
| 65 |
|
| 66 |
def step(self, n_ticks: int = 1) -> None:
|
| 67 |
with obs.span("session.step", **{"session.n_ticks": n_ticks}):
|
|
|
|
| 86 |
def scenario(self):
|
| 87 |
return self.conductor.scenario
|
| 88 |
|
| 89 |
+
# Marks the live, generative session apart from a read-only ``ReplaySession``;
|
| 90 |
+
# the autoplay loop checks this so loading a past run never spends tokens.
|
| 91 |
+
replay = False
|
| 92 |
+
|
| 93 |
@property
|
| 94 |
def events(self):
|
| 95 |
+
# Run-scoped: the ledger is a shared store of *every* run (ADR-0009), so the
|
| 96 |
+
# Show must only ever see the current run's events β otherwise scenario B's
|
| 97 |
+
# stage would replay scenario A's discussion. Every read below (head,
|
| 98 |
+
# snapshot, scrubber) flows from this, so scoping here scopes the whole Show.
|
| 99 |
+
return self.conductor.ledger.events_for_run(self.conductor.run_id)
|
| 100 |
|
| 101 |
@property
|
| 102 |
def head(self) -> int:
|
| 103 |
+
"""The generation-head: number of events in *this run* so far."""
|
| 104 |
+
return len(self.events)
|
| 105 |
|
| 106 |
def has_verdict(self) -> bool:
|
| 107 |
+
"""True once a ``judge.verdict`` event sits in *this run* β the show resolved.
|
| 108 |
+
|
| 109 |
+
Run-scoped (ADR-0009): the ledger is a shared, append-only store of every run,
|
| 110 |
+
so we only consult the current run's events. The Fishbowl autoplay loop calls
|
| 111 |
+
this to auto-pause the timer when the Judge has ruled, so the curtain falls on
|
| 112 |
+
its own (no extra token spend)."""
|
| 113 |
+
return any(e.kind == "judge.verdict" for e in self.conductor.ledger.events_for_run(self.conductor.run_id))
|
| 114 |
+
|
| 115 |
+
def finalize(self, reason: str) -> None:
|
| 116 |
+
"""Close the current run with a ``run.finished`` event (idempotent-safe).
|
| 117 |
+
|
| 118 |
+
On a verdict we derive ``winner`` from the judge's ruling (best-effort: the
|
| 119 |
+
``winner`` payload key when present) and ``winning_model`` from the run.started
|
| 120 |
+
cast map; both fall back to ``None`` when unknown."""
|
| 121 |
+
winner: str | None = None
|
| 122 |
+
winning_model: str | None = None
|
| 123 |
+
run_events = self.conductor.ledger.events_for_run(self.conductor.run_id)
|
| 124 |
+
if reason == "verdict":
|
| 125 |
+
verdict = next((e for e in reversed(run_events) if e.kind == "judge.verdict"), None)
|
| 126 |
+
if verdict is not None:
|
| 127 |
+
winner = verdict.payload.get("winner") or None
|
| 128 |
+
if winner:
|
| 129 |
+
started = next((e for e in run_events if e.kind == "run.started"), None)
|
| 130 |
+
cast = (started.payload.get("cast") or {}) if started is not None else {}
|
| 131 |
+
winning_model = (cast.get(winner) or {}).get("model_endpoint")
|
| 132 |
+
self.conductor.finalize(reason, winner=winner, winning_model=winning_model)
|
| 133 |
|
| 134 |
@property
|
| 135 |
def cast(self) -> list[AgentManifest]:
|
|
|
|
| 181 |
token_ceiling=self.token_ceiling,
|
| 182 |
max_rounds=self.max_rounds,
|
| 183 |
)
|
| 184 |
+
|
| 185 |
+
|
| 186 |
+
class ReplaySession:
|
| 187 |
+
"""A read-only view over one *past* run β the Archive's "Load" target.
|
| 188 |
+
|
| 189 |
+
It exposes the exact read surface the Show renders (``events`` / ``head`` /
|
| 190 |
+
``snapshot`` / ``has_verdict``) so the transport's scrubber and replay just work,
|
| 191 |
+
but it owns no live ``Conductor``: ``step``/``step_one``/``inject`` are no-ops and
|
| 192 |
+
``replay`` is ``True``, so autoplay never generates (no token spend) on a load.
|
| 193 |
+
|
| 194 |
+
The fixed event list is the run's own slice (``events_for_run(run_id)``); the cast
|
| 195 |
+
cards / meters bounds are rebuilt from that run's scenario via the registry, while
|
| 196 |
+
the discussion itself is replayed verbatim from the events.
|
| 197 |
+
"""
|
| 198 |
+
|
| 199 |
+
replay = True
|
| 200 |
+
|
| 201 |
+
def __init__(
|
| 202 |
+
self,
|
| 203 |
+
*,
|
| 204 |
+
run_id: str,
|
| 205 |
+
events: tuple,
|
| 206 |
+
scenario_name: str,
|
| 207 |
+
registry: Registry | None = None,
|
| 208 |
+
tools=None,
|
| 209 |
+
) -> None:
|
| 210 |
+
self.run_id = run_id
|
| 211 |
+
self._events = tuple(events)
|
| 212 |
+
self._scenario_name = scenario_name
|
| 213 |
+
registry = registry or default_registry()
|
| 214 |
+
tools = tools if tools is not None else default_tool_registry()
|
| 215 |
+
scenario = registry.build_scenario(scenario_name, tools=tools)
|
| 216 |
+
self._scenario = scenario
|
| 217 |
+
self._cast = [agent.manifest for agent in scenario.agents]
|
| 218 |
+
self._governor = registry.governor_for(scenario_name)
|
| 219 |
+
obs.log("session.replay", scenario=scenario_name, run_id=run_id, events=len(self._events))
|
| 220 |
+
|
| 221 |
+
# ββ read surface (mirrors FishbowlSession) ββββββββββββββββββββββββββββββββββββ
|
| 222 |
+
|
| 223 |
+
@property
|
| 224 |
+
def events(self):
|
| 225 |
+
return self._events
|
| 226 |
+
|
| 227 |
+
@property
|
| 228 |
+
def head(self) -> int:
|
| 229 |
+
return len(self._events)
|
| 230 |
+
|
| 231 |
+
@property
|
| 232 |
+
def scenario_name(self) -> str:
|
| 233 |
+
return self._scenario_name
|
| 234 |
+
|
| 235 |
+
@property
|
| 236 |
+
def goal(self) -> str:
|
| 237 |
+
return self._scenario.goal
|
| 238 |
+
|
| 239 |
+
@property
|
| 240 |
+
def cast(self) -> list[AgentManifest]:
|
| 241 |
+
return self._cast
|
| 242 |
+
|
| 243 |
+
def has_verdict(self) -> bool:
|
| 244 |
+
return any(e.kind == "judge.verdict" for e in self._events)
|
| 245 |
+
|
| 246 |
+
@property
|
| 247 |
+
def autoplay_tick_cap(self) -> int:
|
| 248 |
+
return self.head
|
| 249 |
+
|
| 250 |
+
# ββ inert lifecycle (a replay never generates) ββββββββββββββββββββββββββββββββ
|
| 251 |
+
|
| 252 |
+
def reset(self, *_args, **_kwargs) -> None: # pragma: no cover - inert by design
|
| 253 |
+
return None
|
| 254 |
+
|
| 255 |
+
def step(self, *_args, **_kwargs) -> None:
|
| 256 |
+
return None
|
| 257 |
+
|
| 258 |
+
def step_one(self, *_args, **_kwargs) -> bool:
|
| 259 |
+
return False
|
| 260 |
+
|
| 261 |
+
def inject(self, *_args, **_kwargs) -> None:
|
| 262 |
+
return None
|
| 263 |
+
|
| 264 |
+
# ββ snapshot ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 265 |
+
|
| 266 |
+
def snapshot(self, k: int | None = None) -> dict:
|
| 267 |
+
events = self._events
|
| 268 |
+
return view_model_at(
|
| 269 |
+
events,
|
| 270 |
+
k if k is not None else len(events),
|
| 271 |
+
self._cast,
|
| 272 |
+
scenario_name=self._scenario_name,
|
| 273 |
+
goal=self.goal,
|
| 274 |
+
governor=None, # no live governor on a replay β meters show recorded text only
|
| 275 |
+
token_ceiling=getattr(self._governor, "max_total_tokens", None),
|
| 276 |
+
max_rounds=getattr(self._governor, "max_turns", None),
|
| 277 |
+
)
|
|
@@ -0,0 +1,244 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Sessions & Archive: per-user run attribution, run-scoped live transcript, and the
|
| 2 |
+
read-only replay the Lab's "Load" drawer hands to the Show.
|
| 3 |
+
|
| 4 |
+
No mocks β the deterministic stub (conftest) drives the cast. These tests use a
|
| 5 |
+
*file-backed* SQLite DB (not the suite's default in-memory ``sqlite://``) because the
|
| 6 |
+
Archive reads through a fresh ``make_ledger()`` each call, and only a file is shared
|
| 7 |
+
across those connections β exactly how the real app's many sessions share one store.
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import pytest
|
| 13 |
+
|
| 14 |
+
from src.core.conductor import Conductor
|
| 15 |
+
from src.core.events import Event, normalize_session_id
|
| 16 |
+
from src.core.ledger_factory import make_ledger
|
| 17 |
+
from src.core.manifest import AgentManifest, ScheduleConfig
|
| 18 |
+
from src.core.registry import default_registry
|
| 19 |
+
from src.scenarios.base import Scenario
|
| 20 |
+
from src.ui.fishbowl import app as fb_app
|
| 21 |
+
from src.ui.fishbowl.archive import list_runs, load_replay, run_card_label
|
| 22 |
+
from src.ui.fishbowl.session import FishbowlSession, ReplaySession
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
@pytest.fixture
|
| 26 |
+
def shared_db(monkeypatch, tmp_path):
|
| 27 |
+
"""Point every ``make_ledger()`` at one on-disk SQLite file (shared across sessions)."""
|
| 28 |
+
url = f"sqlite:///{tmp_path / 'events.db'}"
|
| 29 |
+
monkeypatch.setenv("DATABASE_URL", url)
|
| 30 |
+
return url
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def _two_scenarios() -> tuple[str, str]:
|
| 34 |
+
names = list(default_registry().scenarios)
|
| 35 |
+
assert len(names) >= 2, "need at least two scenarios to test cross-scenario scoping"
|
| 36 |
+
return names[0], names[1]
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def _run(scenario_name: str, session_id: str, *, seed: str = "seed") -> str:
|
| 40 |
+
"""Start a run for *session_id* in *scenario_name*; return its run_id."""
|
| 41 |
+
session = FishbowlSession(scenario_name)
|
| 42 |
+
session.reset(seed, session_id=session_id)
|
| 43 |
+
return session.conductor.run_id
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
# ββ per-user attribution + scoping ββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
class TestRunAttribution:
|
| 50 |
+
def test_run_started_carries_session_id(self, shared_db):
|
| 51 |
+
scenario, _ = _two_scenarios()
|
| 52 |
+
run_id = _run(scenario, "user-A")
|
| 53 |
+
started = next(e for e in make_ledger().events_for_run(run_id) if e.kind == "run.started")
|
| 54 |
+
assert started.payload.get("session_id") == "user-A"
|
| 55 |
+
|
| 56 |
+
def test_no_session_id_omits_the_key(self, shared_db):
|
| 57 |
+
scenario, _ = _two_scenarios()
|
| 58 |
+
session = FishbowlSession(scenario)
|
| 59 |
+
session.reset("seed") # no session_id
|
| 60 |
+
started = next(e for e in make_ledger().events_for_run(session.conductor.run_id) if e.kind == "run.started")
|
| 61 |
+
assert "session_id" not in started.payload
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
class TestLiveTranscriptIsRunScoped:
|
| 65 |
+
def test_session_events_never_bleed_other_runs(self, shared_db):
|
| 66 |
+
# The core fix: the ledger is one shared store of every run, but a live
|
| 67 |
+
# session must only ever see its own run β no cross-run/scenario bleed.
|
| 68 |
+
sx, sy = _two_scenarios()
|
| 69 |
+
a = FishbowlSession(sx)
|
| 70 |
+
a.reset("seed", session_id="user-A")
|
| 71 |
+
a_run = a.conductor.run_id
|
| 72 |
+
|
| 73 |
+
b = FishbowlSession(sy) # a different world, same shared DB
|
| 74 |
+
b.reset("seed", session_id="user-A")
|
| 75 |
+
|
| 76 |
+
# b's ledger physically contains a's events (shared file)...
|
| 77 |
+
assert any(e.run_id == a_run for e in b.conductor.ledger.events)
|
| 78 |
+
# ...yet b's *view* shows only b's run, and a's only a's.
|
| 79 |
+
assert {e.run_id for e in b.events} == {b.conductor.run_id}
|
| 80 |
+
assert {e.run_id for e in a.events} == {a_run}
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
# ββ the Archive list ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
class TestListRuns:
|
| 87 |
+
def test_filters_to_my_runs_in_this_world(self, shared_db):
|
| 88 |
+
sx, sy = _two_scenarios()
|
| 89 |
+
mine_x = _run(sx, "me")
|
| 90 |
+
_run(sx, "someone-else") # same world, different user
|
| 91 |
+
_run(sy, "me") # my run, different world
|
| 92 |
+
|
| 93 |
+
rows = list_runs(sx, "me")
|
| 94 |
+
assert [r.run_id for r in rows] == [mine_x]
|
| 95 |
+
assert rows[0].session_id == "me"
|
| 96 |
+
assert rows[0].scenario == sx
|
| 97 |
+
|
| 98 |
+
def test_newest_first(self, shared_db):
|
| 99 |
+
sx, _ = _two_scenarios()
|
| 100 |
+
first = _run(sx, "me", seed="one")
|
| 101 |
+
second = _run(sx, "me", seed="two")
|
| 102 |
+
rows = list_runs(sx, "me")
|
| 103 |
+
assert [r.run_id for r in rows] == [second, first]
|
| 104 |
+
|
| 105 |
+
def test_empty_without_session_id(self, shared_db):
|
| 106 |
+
sx, _ = _two_scenarios()
|
| 107 |
+
_run(sx, "me")
|
| 108 |
+
assert list_runs(sx, None) == []
|
| 109 |
+
assert list_runs(sx, "") == []
|
| 110 |
+
|
| 111 |
+
def test_run_card_label_is_a_single_line(self, shared_db):
|
| 112 |
+
sx, _ = _two_scenarios()
|
| 113 |
+
_run(sx, "me")
|
| 114 |
+
label = run_card_label(list_runs(sx, "me")[0])
|
| 115 |
+
assert "\n" not in label and label.startswith("βΆ")
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
# ββ read-only replay βββββββββββββββββββββββββββββββββββββοΏ½οΏ½οΏ½βββββββββββββββββββββββββ
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
class TestReplay:
|
| 122 |
+
def test_load_replay_reconstructs_the_run(self, shared_db):
|
| 123 |
+
sx, _ = _two_scenarios()
|
| 124 |
+
run_id = _run(sx, "me")
|
| 125 |
+
expected = make_ledger().events_for_run(run_id)
|
| 126 |
+
|
| 127 |
+
replay = load_replay(run_id)
|
| 128 |
+
assert isinstance(replay, ReplaySession)
|
| 129 |
+
assert replay.replay is True
|
| 130 |
+
assert [e.id for e in replay.events] == [e.id for e in expected]
|
| 131 |
+
assert replay.scenario_name == sx
|
| 132 |
+
# The snapshot renders without a live conductor.
|
| 133 |
+
vm = replay.snapshot()
|
| 134 |
+
assert vm["total"] == len(expected)
|
| 135 |
+
|
| 136 |
+
def test_load_replay_unknown_run_is_none(self, shared_db):
|
| 137 |
+
assert load_replay("no-such-run") is None
|
| 138 |
+
assert load_replay("") is None
|
| 139 |
+
|
| 140 |
+
def test_replay_session_is_inert(self, shared_db):
|
| 141 |
+
sx, _ = _two_scenarios()
|
| 142 |
+
replay = load_replay(_run(sx, "me"))
|
| 143 |
+
head_before = replay.head
|
| 144 |
+
assert replay.step_one() is False
|
| 145 |
+
replay.step()
|
| 146 |
+
replay.inject("nudge")
|
| 147 |
+
assert replay.head == head_before # nothing generated
|
| 148 |
+
|
| 149 |
+
def test_advance_replays_prefix_then_stops(self, shared_db):
|
| 150 |
+
sx, _ = _two_scenarios()
|
| 151 |
+
replay = load_replay(_run(sx, "me"))
|
| 152 |
+
assert replay.head >= 1
|
| 153 |
+
# From 0 it walks forward through the recorded prefix without generating.
|
| 154 |
+
k, ticks, reason = fb_app.advance_one_tick(replay, 0, 0)
|
| 155 |
+
assert (k, reason) == (1, None)
|
| 156 |
+
# At the head it stops cleanly (no token spend, no backstop trip).
|
| 157 |
+
k, ticks, reason = fb_app.advance_one_tick(replay, replay.head, 0)
|
| 158 |
+
assert k == replay.head and reason is not None
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
# ββ session hardening: envelope stamping, normalization, context scoping ββββββββββ
|
| 162 |
+
|
| 163 |
+
|
| 164 |
+
class _ProbeAgent:
|
| 165 |
+
"""Records the run_ids of every event the conductor hands it as context."""
|
| 166 |
+
|
| 167 |
+
name = "probe"
|
| 168 |
+
manifest = AgentManifest(name="probe", persona="p", may_emit=["agent.spoke"], schedule=ScheduleConfig(tick_every=1))
|
| 169 |
+
|
| 170 |
+
def __init__(self) -> None:
|
| 171 |
+
self.last_usage: dict = {}
|
| 172 |
+
self.seen_run_ids: set[str] = set()
|
| 173 |
+
|
| 174 |
+
def act(self, run_id, turn, projection, recent_events) -> Event:
|
| 175 |
+
self.seen_run_ids.update(e.run_id for e in recent_events)
|
| 176 |
+
return Event(run_id=run_id, turn=turn, kind="agent.spoke", actor="probe", payload={"text": "t"})
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
def _probe_conductor() -> tuple[Conductor, _ProbeAgent]:
|
| 180 |
+
probe = _ProbeAgent()
|
| 181 |
+
scenario = Scenario(name="probe-world", default_seed="seed", agents=(probe,))
|
| 182 |
+
return Conductor(scenario, ledger=make_ledger()), probe
|
| 183 |
+
|
| 184 |
+
|
| 185 |
+
class TestNormalizeSessionId:
|
| 186 |
+
def test_accepts_minted_shapes(self):
|
| 187 |
+
assert normalize_session_id("3b456aab-4ac6-46d6-981a-ff10a8f25fdb") is not None
|
| 188 |
+
assert normalize_session_id("sess-abc123xyz") == "sess-abc123xyz"
|
| 189 |
+
assert normalize_session_id(" padded-id ") == "padded-id"
|
| 190 |
+
|
| 191 |
+
def test_rejects_untrusted_garbage(self):
|
| 192 |
+
assert normalize_session_id(None) is None
|
| 193 |
+
assert normalize_session_id("") is None
|
| 194 |
+
assert normalize_session_id("a" * 65) is None # over-long
|
| 195 |
+
assert normalize_session_id("has spaces") is None
|
| 196 |
+
assert normalize_session_id("<script>alert(1)</script>") is None
|
| 197 |
+
|
| 198 |
+
def test_conductor_normalizes_at_the_boundary(self, shared_db):
|
| 199 |
+
c, _probe = _probe_conductor()
|
| 200 |
+
c.reset("seed", session_id="<not a valid id>")
|
| 201 |
+
assert c.session_id is None
|
| 202 |
+
started = next(e for e in c.ledger.events_for_run(c.run_id) if e.kind == "run.started")
|
| 203 |
+
assert "session_id" not in started.payload and started.session_id is None
|
| 204 |
+
|
| 205 |
+
|
| 206 |
+
class TestEnvelopeStamping:
|
| 207 |
+
def test_every_event_in_a_run_carries_the_session_id(self, shared_db):
|
| 208 |
+
c, _probe = _probe_conductor()
|
| 209 |
+
c.reset("seed", session_id="user-A")
|
| 210 |
+
c.step(2) # genesis + agent turns β several event kinds
|
| 211 |
+
events = c.ledger.events_for_run(c.run_id)
|
| 212 |
+
assert len(events) >= 3
|
| 213 |
+
assert all(e.session_id == "user-A" for e in events)
|
| 214 |
+
|
| 215 |
+
def test_session_id_round_trips_through_sql(self, shared_db):
|
| 216 |
+
c, _probe = _probe_conductor()
|
| 217 |
+
c.reset("seed", session_id="user-A")
|
| 218 |
+
c.step(1)
|
| 219 |
+
# A fresh connection re-reads rows from disk β the envelope must survive.
|
| 220 |
+
reread = make_ledger().events_for_run(c.run_id)
|
| 221 |
+
assert reread and all(e.session_id == "user-A" for e in reread)
|
| 222 |
+
|
| 223 |
+
def test_unattributed_run_stays_null(self, shared_db):
|
| 224 |
+
c, _probe = _probe_conductor()
|
| 225 |
+
c.reset("seed") # headless run, no session
|
| 226 |
+
c.step(1)
|
| 227 |
+
assert all(e.session_id is None for e in c.ledger.events_for_run(c.run_id))
|
| 228 |
+
|
| 229 |
+
|
| 230 |
+
class TestAgentContextIsRunScoped:
|
| 231 |
+
def test_agents_never_see_other_runs_in_recent_events(self, shared_db):
|
| 232 |
+
# Run A fills the shared store with another user's discussion.
|
| 233 |
+
a, _ = _probe_conductor()
|
| 234 |
+
a.reset("seed", session_id="user-A")
|
| 235 |
+
a.step(2)
|
| 236 |
+
a_run = a.run_id
|
| 237 |
+
|
| 238 |
+
# Run B (fresh conductor, same DB) β its agent's context must be B-only.
|
| 239 |
+
b, probe_b = _probe_conductor()
|
| 240 |
+
b.reset("seed", session_id="user-B")
|
| 241 |
+
b.step(2)
|
| 242 |
+
|
| 243 |
+
assert any(e.run_id == a_run for e in b.ledger.events) # store IS shared...
|
| 244 |
+
assert probe_b.seen_run_ids == {b.run_id} # ...but the agent's context is not
|
|
@@ -83,7 +83,7 @@ class _RaisingIndex:
|
|
| 83 |
def index(self, events):
|
| 84 |
raise RuntimeError("backend down")
|
| 85 |
|
| 86 |
-
def search(self, query, k):
|
| 87 |
raise RuntimeError("backend down")
|
| 88 |
|
| 89 |
|
|
|
|
| 83 |
def index(self, events):
|
| 84 |
raise RuntimeError("backend down")
|
| 85 |
|
| 86 |
+
def search(self, query, k, run_id=None):
|
| 87 |
raise RuntimeError("backend down")
|
| 88 |
|
| 89 |
|
|
@@ -49,15 +49,18 @@ class _FakeIndex:
|
|
| 49 |
def __init__(self) -> None:
|
| 50 |
self.store: dict[str, Event] = {}
|
| 51 |
self.add_calls: list[str] = []
|
|
|
|
| 52 |
|
| 53 |
def index(self, events: tuple[Event, ...]) -> None:
|
| 54 |
for e in events:
|
| 55 |
self.add_calls.append(e.id)
|
| 56 |
self.store[e.id] = e # upsert by id β idempotent
|
| 57 |
|
| 58 |
-
def search(self, query: str, k: int) -> list[Event]:
|
|
|
|
| 59 |
q = set(query.lower().split())
|
| 60 |
-
|
|
|
|
| 61 |
scored.sort(key=lambda t: t[0], reverse=True)
|
| 62 |
return [e for _, e in scored[:k]]
|
| 63 |
|
|
@@ -115,6 +118,22 @@ class TestSalienceUsesIndex:
|
|
| 115 |
mem.visible((mine, theirs, glob), current_turn=2, query="stage")
|
| 116 |
assert set(idx.store) == {"mine", "glob"} # 'theirs' never indexed
|
| 117 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
def test_recency_still_applies_with_index(self):
|
| 119 |
"""Relevance is one term; recency must still separate equally-relevant
|
| 120 |
events so the index does not flatten the composite score."""
|
|
|
|
| 49 |
def __init__(self) -> None:
|
| 50 |
self.store: dict[str, Event] = {}
|
| 51 |
self.add_calls: list[str] = []
|
| 52 |
+
self.search_run_ids: list[str | None] = []
|
| 53 |
|
| 54 |
def index(self, events: tuple[Event, ...]) -> None:
|
| 55 |
for e in events:
|
| 56 |
self.add_calls.append(e.id)
|
| 57 |
self.store[e.id] = e # upsert by id β idempotent
|
| 58 |
|
| 59 |
+
def search(self, query: str, k: int, run_id: str | None = None) -> list[Event]:
|
| 60 |
+
self.search_run_ids.append(run_id)
|
| 61 |
q = set(query.lower().split())
|
| 62 |
+
pool = [e for e in self.store.values() if run_id is None or e.run_id == run_id]
|
| 63 |
+
scored = [(len(q & set(str(e.payload.get("text", "")).lower().split())), e) for e in pool]
|
| 64 |
scored.sort(key=lambda t: t[0], reverse=True)
|
| 65 |
return [e for _, e in scored[:k]]
|
| 66 |
|
|
|
|
| 118 |
mem.visible((mine, theirs, glob), current_turn=2, query="stage")
|
| 119 |
assert set(idx.store) == {"mine", "glob"} # 'theirs' never indexed
|
| 120 |
|
| 121 |
+
def test_search_is_scoped_to_the_candidates_run(self):
|
| 122 |
+
"""The index spans every run in the shared store; recall must be scoped to
|
| 123 |
+
the run the candidates came from, so one show's (or one user's) discussion
|
| 124 |
+
never crowds another's relevance budget."""
|
| 125 |
+
idx = _FakeIndex()
|
| 126 |
+
# The index already holds an event from ANOTHER run that matches the query.
|
| 127 |
+
foreign = Event(run_id="other-run", turn=1, kind="world.observed", actor="n", payload={"text": "beacon glow"})
|
| 128 |
+
idx.index((foreign,))
|
| 129 |
+
|
| 130 |
+
ours = _event("world.observed", turn=2, text="beacon glow signal", eid="ours") # run_id="r"
|
| 131 |
+
mem = SalienceMemory("x", top_k=2, index=idx)
|
| 132 |
+
recalled = mem.visible((ours,), current_turn=3, query="beacon glow")
|
| 133 |
+
|
| 134 |
+
assert idx.search_run_ids == ["r"] # the search was run-scoped...
|
| 135 |
+
assert [e.id for e in recalled] == ["ours"] # ...and the foreign hit never surfaced
|
| 136 |
+
|
| 137 |
def test_recency_still_applies_with_index(self):
|
| 138 |
"""Relevance is one term; recency must still separate equally-relevant
|
| 139 |
events so the index does not flatten the composite score."""
|
|
@@ -0,0 +1,342 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Run history & lifecycle: ``run.started`` enrichment, ``run.finished`` emission,
|
| 2 |
+
non-destructive reset, the run-scoped read surface, and multi-run indexing.
|
| 3 |
+
|
| 4 |
+
No mocks β the deterministic stub (conftest) and a couple of tiny in-process agents
|
| 5 |
+
drive everything, mirroring the idiom in ``test_conductor.py``.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import pytest
|
| 11 |
+
|
| 12 |
+
from src.core.conductor import Conductor
|
| 13 |
+
from src.core.events import Event
|
| 14 |
+
from src.core.governor import BudgetExceeded, Governor
|
| 15 |
+
from src.core.ledger import Ledger
|
| 16 |
+
from src.core.manifest import AgentManifest, ScheduleConfig
|
| 17 |
+
from src.core.projections import rebuild_stage
|
| 18 |
+
from src.core.run_index import RunSummary, index_runs, index_runs_from_ledger
|
| 19 |
+
from src.core.sqlalchemy_ledger import SqlAlchemyLedger
|
| 20 |
+
from src.scenarios.base import Scenario
|
| 21 |
+
from src.ui.fishbowl.session import FishbowlSession
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
# ββ tiny in-process agents ββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
class _JudgingAgent:
|
| 28 |
+
"""Rules a verdict every tick β stands in for the live Judge so a run resolves."""
|
| 29 |
+
|
| 30 |
+
name = "judge"
|
| 31 |
+
manifest = AgentManifest(
|
| 32 |
+
name="judge",
|
| 33 |
+
persona="p",
|
| 34 |
+
may_emit=["judge.verdict"],
|
| 35 |
+
schedule=ScheduleConfig(tick_every=1),
|
| 36 |
+
model_endpoint="model://judge-endpoint",
|
| 37 |
+
model_profile="strong",
|
| 38 |
+
)
|
| 39 |
+
|
| 40 |
+
def __init__(self) -> None:
|
| 41 |
+
self.last_usage: dict = {}
|
| 42 |
+
|
| 43 |
+
def act(self, run_id, turn, projection, recent_events) -> Event:
|
| 44 |
+
return Event(
|
| 45 |
+
run_id=run_id,
|
| 46 |
+
turn=turn,
|
| 47 |
+
kind="judge.verdict",
|
| 48 |
+
actor="judge",
|
| 49 |
+
payload={"text": "the verdict", "winner": "judge"},
|
| 50 |
+
)
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
class _SpeakingAgent:
|
| 54 |
+
name = "speaker"
|
| 55 |
+
manifest = AgentManifest(
|
| 56 |
+
name="speaker", persona="p", may_emit=["agent.spoke"], schedule=ScheduleConfig(tick_every=1)
|
| 57 |
+
)
|
| 58 |
+
|
| 59 |
+
def __init__(self) -> None:
|
| 60 |
+
self.last_usage: dict = {}
|
| 61 |
+
|
| 62 |
+
def act(self, run_id, turn, projection, recent_events) -> Event:
|
| 63 |
+
return Event(run_id=run_id, turn=turn, kind="agent.spoke", actor="speaker", payload={"text": "hi"})
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def _verdict_scenario() -> Scenario:
|
| 67 |
+
return Scenario(name="verdict-world", default_seed="seed", agents=(_JudgingAgent(),))
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
# ββ run.started enrichment ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
class TestRunStartedPayload:
|
| 74 |
+
def test_run_started_carries_cast_model_map_and_scenario(self):
|
| 75 |
+
c = Conductor(scenario=_verdict_scenario())
|
| 76 |
+
c.reset("seed")
|
| 77 |
+
started = next(e for e in c.ledger.events_for_run(c.run_id) if e.kind == "run.started")
|
| 78 |
+
assert started.payload["scenario"] == "verdict-world"
|
| 79 |
+
cast = started.payload["cast"]
|
| 80 |
+
assert cast["judge"] == {"model_endpoint": "model://judge-endpoint", "model_profile": "strong"}
|
| 81 |
+
|
| 82 |
+
def test_run_started_reports_unbound_agent_without_manifest_fields(self):
|
| 83 |
+
c = Conductor(scenario=Scenario(name="s", default_seed="seed", agents=(_SpeakingAgent(),)))
|
| 84 |
+
c.reset("seed")
|
| 85 |
+
started = next(e for e in c.ledger.events_for_run(c.run_id) if e.kind == "run.started")
|
| 86 |
+
assert started.payload["cast"]["speaker"]["model_endpoint"] is None
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
# ββ run.finished emission βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
class TestRunFinished:
|
| 93 |
+
def test_finished_on_verdict_via_session_finalize(self):
|
| 94 |
+
session = FishbowlSession("thousand-token-wood")
|
| 95 |
+
session.reset()
|
| 96 |
+
# Drive until the live Judge rules (the stub cast resolves quickly).
|
| 97 |
+
for _ in range(session.autoplay_tick_cap):
|
| 98 |
+
if session.has_verdict():
|
| 99 |
+
break
|
| 100 |
+
session.step()
|
| 101 |
+
assert session.has_verdict(), "the stub cast should reach a verdict"
|
| 102 |
+
|
| 103 |
+
session.finalize("verdict")
|
| 104 |
+
finished = [
|
| 105 |
+
e for e in session.conductor.ledger.events_for_run(session.conductor.run_id) if e.kind == "run.finished"
|
| 106 |
+
]
|
| 107 |
+
assert len(finished) == 1
|
| 108 |
+
assert finished[0].payload["reason"] == "verdict"
|
| 109 |
+
|
| 110 |
+
def test_finished_on_budget_trip_carries_budget_reason(self):
|
| 111 |
+
# A per-turn cap of 1 trips BudgetExceeded on the SECOND agent; the conductor
|
| 112 |
+
# must close the run with reason "budget" before the stop propagates.
|
| 113 |
+
scenario = Scenario(name="s", default_seed="seed", agents=(_SpeakingAgent(), _SpeakingAgent()))
|
| 114 |
+
c = Conductor(scenario=scenario, governor=Governor(max_calls_per_turn=1))
|
| 115 |
+
c.reset("seed")
|
| 116 |
+
with pytest.raises(BudgetExceeded):
|
| 117 |
+
c.step()
|
| 118 |
+
finished = [e for e in c.ledger.events_for_run(c.run_id) if e.kind == "run.finished"]
|
| 119 |
+
assert len(finished) == 1
|
| 120 |
+
assert finished[0].payload["reason"] == "budget"
|
| 121 |
+
|
| 122 |
+
def test_finalize_is_idempotent(self):
|
| 123 |
+
c = Conductor(scenario=_verdict_scenario())
|
| 124 |
+
c.reset("seed")
|
| 125 |
+
first = c.finalize("user_stop")
|
| 126 |
+
second = c.finalize("verdict") # must NOT emit a second run.finished
|
| 127 |
+
assert first is not None and second is not None
|
| 128 |
+
assert first.id == second.id
|
| 129 |
+
finished = [e for e in c.ledger.events_for_run(c.run_id) if e.kind == "run.finished"]
|
| 130 |
+
assert len(finished) == 1
|
| 131 |
+
assert finished[0].payload["reason"] == "user_stop"
|
| 132 |
+
|
| 133 |
+
def test_session_finalize_derives_winner_and_model_from_verdict(self):
|
| 134 |
+
scenario = _verdict_scenario()
|
| 135 |
+
# Build a session-like conductor manually so we control the cast verdict.
|
| 136 |
+
c = Conductor(scenario=scenario)
|
| 137 |
+
c.reset("seed")
|
| 138 |
+
c.step() # judge rules: winner="judge"
|
| 139 |
+
# Reproduce FishbowlSession.finalize's derivation against this conductor.
|
| 140 |
+
run_events = c.ledger.events_for_run(c.run_id)
|
| 141 |
+
verdict = next(e for e in reversed(run_events) if e.kind == "judge.verdict")
|
| 142 |
+
winner = verdict.payload.get("winner")
|
| 143 |
+
started = next(e for e in run_events if e.kind == "run.started")
|
| 144 |
+
winning_model = (started.payload["cast"].get(winner) or {}).get("model_endpoint")
|
| 145 |
+
finished = c.finalize("verdict", winner=winner, winning_model=winning_model)
|
| 146 |
+
assert finished.payload["winner"] == "judge"
|
| 147 |
+
assert finished.payload["winning_model"] == "model://judge-endpoint"
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
# ββ non-destructive reset βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
class TestNonDestructiveReset:
|
| 154 |
+
def test_reset_starts_new_run_and_keeps_prior_run_events(self):
|
| 155 |
+
c = Conductor(scenario=_verdict_scenario())
|
| 156 |
+
c.reset("seed-a")
|
| 157 |
+
old_run = c.run_id
|
| 158 |
+
c.step()
|
| 159 |
+
old_events = c.ledger.events_for_run(old_run)
|
| 160 |
+
assert old_events, "the first run produced events"
|
| 161 |
+
|
| 162 |
+
c.reset("seed-b")
|
| 163 |
+
new_run = c.run_id
|
| 164 |
+
assert new_run != old_run, "reset mints a NEW run_id"
|
| 165 |
+
# Prior run's events survive on the shared, append-only ledger.
|
| 166 |
+
assert c.ledger.events_for_run(old_run) == old_events
|
| 167 |
+
# The new run only sees its own run.started/genesis so far.
|
| 168 |
+
new_events = c.ledger.events_for_run(new_run)
|
| 169 |
+
assert new_events and all(e.run_id == new_run for e in new_events)
|
| 170 |
+
assert any(e.kind == "run.started" for e in new_events)
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
# ββ run-scoped read surface βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
def _multi_run_ledger(ledger: Ledger) -> dict[str, list[Event]]:
|
| 177 |
+
"""Append two interleaved runs to *ledger*; return {run_id: events} oracle."""
|
| 178 |
+
expected: dict[str, list[Event]] = {"run-A": [], "run-B": []}
|
| 179 |
+
plan = [
|
| 180 |
+
("run-A", 0, "run.started"),
|
| 181 |
+
("run-B", 0, "run.started"),
|
| 182 |
+
("run-A", 1, "agent.spoke"),
|
| 183 |
+
("run-A", 1, "judge.verdict"),
|
| 184 |
+
("run-B", 1, "agent.spoke"),
|
| 185 |
+
("run-A", 2, "run.finished"),
|
| 186 |
+
]
|
| 187 |
+
for run_id, turn, kind in plan:
|
| 188 |
+
payload = {"seed": run_id} if kind == "run.started" else {"text": "t"}
|
| 189 |
+
e = Event(run_id=run_id, turn=turn, kind=kind, actor="x", payload=payload)
|
| 190 |
+
ledger.append(e)
|
| 191 |
+
expected[run_id].append(e)
|
| 192 |
+
return expected
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
class TestRunScopedReads:
|
| 196 |
+
@pytest.mark.parametrize("ledger_factory", [Ledger, lambda: SqlAlchemyLedger("sqlite://")])
|
| 197 |
+
def test_events_for_run_and_runs(self, ledger_factory):
|
| 198 |
+
ledger = ledger_factory()
|
| 199 |
+
expected = _multi_run_ledger(ledger)
|
| 200 |
+
|
| 201 |
+
# runs() reports distinct run_ids in first-seen order.
|
| 202 |
+
assert ledger.runs() == ("run-A", "run-B")
|
| 203 |
+
|
| 204 |
+
for run_id, events in expected.items():
|
| 205 |
+
got = ledger.events_for_run(run_id)
|
| 206 |
+
assert [e.id for e in got] == [e.id for e in events]
|
| 207 |
+
assert [e.kind for e in got] == [e.kind for e in events]
|
| 208 |
+
|
| 209 |
+
def test_rebuild_stage_ignores_other_runs(self):
|
| 210 |
+
ledger = Ledger()
|
| 211 |
+
_multi_run_ledger(ledger)
|
| 212 |
+
proj = rebuild_stage(ledger.events, "run-B")
|
| 213 |
+
# run-B never emitted a verdict; run-A did β scoping must not bleed it in.
|
| 214 |
+
assert proj.judge_notes == []
|
| 215 |
+
|
| 216 |
+
def test_rebuild_stage_unscoped_sees_everything(self):
|
| 217 |
+
ledger = Ledger()
|
| 218 |
+
_multi_run_ledger(ledger)
|
| 219 |
+
scoped = rebuild_stage(ledger.events, "run-B")
|
| 220 |
+
unscoped = rebuild_stage(ledger.events)
|
| 221 |
+
# The unscoped projection folds in run-A's verdict; the scoped one does not.
|
| 222 |
+
assert unscoped.judge_notes != []
|
| 223 |
+
assert scoped.judge_notes == []
|
| 224 |
+
|
| 225 |
+
|
| 226 |
+
# ββ run index: oracle vs. SQL implementation ββββββββββββββββββββββββββββββββββββββ
|
| 227 |
+
|
| 228 |
+
|
| 229 |
+
def _index_oracle(events: tuple[Event, ...]) -> dict[str, list[str]]:
|
| 230 |
+
"""Reference per-run index: {run_id: [event_id, ...]} in first-seen/offset order."""
|
| 231 |
+
index: dict[str, list[str]] = {}
|
| 232 |
+
for e in events:
|
| 233 |
+
index.setdefault(e.run_id, []).append(e.id)
|
| 234 |
+
return index
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
class TestRunIndex:
|
| 238 |
+
def test_sql_run_index_matches_oracle(self):
|
| 239 |
+
sql = SqlAlchemyLedger("sqlite://")
|
| 240 |
+
mem = Ledger()
|
| 241 |
+
# Feed the SAME multi-run stream into a plain list to build the oracle.
|
| 242 |
+
plan = [
|
| 243 |
+
("r1", "run.started"),
|
| 244 |
+
("r2", "run.started"),
|
| 245 |
+
("r1", "agent.spoke"),
|
| 246 |
+
("r3", "run.started"),
|
| 247 |
+
("r2", "agent.spoke"),
|
| 248 |
+
("r1", "run.finished"),
|
| 249 |
+
("r3", "agent.spoke"),
|
| 250 |
+
("r2", "run.finished"),
|
| 251 |
+
]
|
| 252 |
+
stream: list[Event] = []
|
| 253 |
+
for run_id, kind in plan:
|
| 254 |
+
e = Event(run_id=run_id, turn=0, kind=kind, actor="x", payload={"text": "t"})
|
| 255 |
+
sql.append(e)
|
| 256 |
+
mem.append(e)
|
| 257 |
+
stream.append(e)
|
| 258 |
+
|
| 259 |
+
oracle = _index_oracle(tuple(stream))
|
| 260 |
+
|
| 261 |
+
# runs() must agree with the oracle's first-seen run order.
|
| 262 |
+
assert list(sql.runs()) == list(oracle.keys())
|
| 263 |
+
assert list(mem.runs()) == list(oracle.keys())
|
| 264 |
+
|
| 265 |
+
# Per-run event ordering must match the oracle on the SQL backend.
|
| 266 |
+
for run_id, ids in oracle.items():
|
| 267 |
+
assert [e.id for e in sql.events_for_run(run_id)] == ids
|
| 268 |
+
assert [e.id for e in mem.events_for_run(run_id)] == ids
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
# ββ run_index module: index_runs oracle vs. index_runs_from_ledger (SQL path) βββββ
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
def _bookended_runs() -> list[Event]:
|
| 275 |
+
"""A two-run stream with full run.started/run.finished payloads, interleaved."""
|
| 276 |
+
return [
|
| 277 |
+
Event(
|
| 278 |
+
run_id="r1",
|
| 279 |
+
turn=0,
|
| 280 |
+
kind="run.started",
|
| 281 |
+
actor="conductor",
|
| 282 |
+
payload={
|
| 283 |
+
"seed": "acorn",
|
| 284 |
+
"goal": "g",
|
| 285 |
+
"scenario": "thousand_token_wood",
|
| 286 |
+
"cast": {"judge": {"model_endpoint": "model://j", "model_profile": "strong"}},
|
| 287 |
+
},
|
| 288 |
+
),
|
| 289 |
+
Event(
|
| 290 |
+
run_id="r2",
|
| 291 |
+
turn=0,
|
| 292 |
+
kind="run.started",
|
| 293 |
+
actor="conductor",
|
| 294 |
+
payload={"seed": "burrow", "scenario": "fishbowl", "cast": {}},
|
| 295 |
+
),
|
| 296 |
+
Event(run_id="r1", turn=1, kind="agent.spoke", actor="judge", payload={"text": "t"}),
|
| 297 |
+
Event(
|
| 298 |
+
run_id="r1",
|
| 299 |
+
turn=2,
|
| 300 |
+
kind="run.finished",
|
| 301 |
+
actor="conductor",
|
| 302 |
+
payload={
|
| 303 |
+
"reason": "verdict",
|
| 304 |
+
"winner": "judge",
|
| 305 |
+
"winning_model": "model://j",
|
| 306 |
+
"turns": 2,
|
| 307 |
+
"tokens": 1234,
|
| 308 |
+
},
|
| 309 |
+
),
|
| 310 |
+
# r2 starts but never finishes β its finished_* fields stay None/0.
|
| 311 |
+
Event(run_id="r2", turn=1, kind="agent.spoke", actor="x", payload={"text": "t"}),
|
| 312 |
+
]
|
| 313 |
+
|
| 314 |
+
|
| 315 |
+
class TestRunIndexModule:
|
| 316 |
+
def test_index_runs_folds_bookend_events(self):
|
| 317 |
+
summaries = index_runs(_bookended_runs())
|
| 318 |
+
assert [s.run_id for s in summaries] == ["r1", "r2"] # first-seen order
|
| 319 |
+
|
| 320 |
+
r1, r2 = summaries
|
| 321 |
+
assert isinstance(r1, RunSummary)
|
| 322 |
+
assert (r1.scenario, r1.seed) == ("thousand_token_wood", "acorn")
|
| 323 |
+
assert r1.cast["judge"].model_endpoint == "model://j"
|
| 324 |
+
assert r1.cast["judge"].model_profile == "strong"
|
| 325 |
+
assert (r1.reason, r1.winner, r1.winning_model) == ("verdict", "judge", "model://j")
|
| 326 |
+
assert (r1.turns, r1.tokens) == (2, 1234)
|
| 327 |
+
assert r1.started_at is not None and r1.finished_at is not None
|
| 328 |
+
|
| 329 |
+
# r2 started but never finished β terminal fields stay at their defaults.
|
| 330 |
+
assert (r2.scenario, r2.seed) == ("fishbowl", "burrow")
|
| 331 |
+
assert r2.reason is None and r2.winner is None
|
| 332 |
+
assert (r2.turns, r2.tokens) == (0, 0)
|
| 333 |
+
assert r2.finished_at is None
|
| 334 |
+
|
| 335 |
+
@pytest.mark.parametrize("make_ledger", [Ledger, lambda: SqlAlchemyLedger("sqlite://")])
|
| 336 |
+
def test_from_ledger_matches_pure_oracle(self, make_ledger):
|
| 337 |
+
events = _bookended_runs()
|
| 338 |
+
ledger = make_ledger()
|
| 339 |
+
for e in events:
|
| 340 |
+
ledger.append(e)
|
| 341 |
+
# The indexed-query path must produce exactly what the pure oracle does.
|
| 342 |
+
assert index_runs_from_ledger(ledger) == index_runs(events)
|