feat: Add Hall of Fame leaderboard backed by a dedicated results table
feat: Add Hall of Fame leaderboard backed by a dedicated results table
agharsallah+1: Codex
c952d77
Add a competitive-results leaderboard as a dedicated, durable table
(leaderboard_entries) detached from the event ledger: the events log stays
the run trace, while one denormalised scoreboard row per decided run records
who won on which model. A row is written only when a run is finished, has a
winner, names a concrete winning model, and ran a competitive scenario; the
run_id links each row back to its full trace for replay.
- src/core/leaderboard_store.py: LeaderboardEntry + LeaderboardStore
(SQLAlchemy, separate table in the same database as the event store),
build_entry write gate, memoised make_leaderboard_store factory.
- src/core/leaderboard.py: model / agent / fairness / sessions / headline
aggregations over stored entries, with a per-seat fairness rollup.
- src/ui/fishbowl/session.py: record a scoreboard row on finalize.
- src/ui/fishbowl/hall_of_fame.py + app.py: the "Hall of Fame" tab —
champion headline, model podium with win-rate bars, sessions with replay,
and t…
feat(media): introduce MediaRouter and stubs for image and speech generation
agharsallah
8400d8c
- Implement MediaRouter to resolve media capabilities to providers, supporting both live and offline modes.
- Add StubImageProvider and StubSpeechProvider for deterministic offline media generation.
- Create tools for image rendering and text-to-speech synthesis, integrating with the ToolRegistry.
- Enhance the commentator (rafters-critic) to generate and render media alongside commentary.
- Update UI components to display generated images and audio clips in the feed.
- Ensure tests cover the new media capabilities, including deterministic outputs and capability enforcement.
feat: Implement early termination for the judge in Twenty Sprouts game and enhance event handling
agharsallah
0460922
feat: Implement audience-only secret badge for Twenty Sprouts game
agharsallah
f6566bb
- Added `_secret_badge` function to render a hidden-word run's secret for the audience.
- Updated `_core` function to include the secret badge in the HTML output.
- Introduced `peek_next_actor_name` method in `FishbowlSession` to provide the next actor's name without advancing the run.
- Modified `ReplaySession` to handle `peek_next_actor_name` appropriately.
- Refactored `build_show` to simplify the show controls, removing unnecessary buttons and adding play/restart functionality.
- Enhanced `view_model_at` to include the secret and its holder for the audience's view.
- Added tests for the new functionality, ensuring the secret is handled correctly and does not leak into agent prompts.
- Created a new test suite for Twenty Sprouts gameplay to validate the secret handling and game mechanics.
feat: implement winner celebration overlay and judging mechanics
agharsallah
6432854
- Added `render_winner` function to create a celebratory overlay for the winning agent or team.
- Integrated winner rendering into the main application flow, allowing for a modular display of victory.
- Introduced `has_judge` and `force_verdict` methods in `FishbowlSession` to manage judging logic.
- Updated session handling to allow for forced verdicts, even when the budget is spent.
- Enhanced CSS styles for the winner celebration, including confetti and trophy animations.
- Created tests for judging mechanics and winner rendering to ensure correct functionality across scenarios.
Merge remote-tracking branch 'origin/main' into worktree-scenarios
feat: make every scenario arena-grade with a competition contract
agharsallah+1: Codex
0d0c561
Add a validated `competition` block (versus|judged|none) to scenarios,
stamped onto run.started so runs are self-describing and winners attribute
to models. Winners become data, not prose: judges emit a `winner` field
that the reusable JudgedCompetition handler validates against the live cast
(repairing deterministically so the offline stub always names a real
player); ground-truth judges (SpyHost, SproutJudge) decide in code.
Audit the five existing scenarios (The Steeped versus+teams with code-scored
winner; Mystery Roots judged; Open Table gains a table-judge; Thousand Token
Wood and Oracle Grove marked none) and add three competitive scenarios:
Debate Duel and Beat Battle (symmetric seats, different models) and Twenty
Sprouts (code-dealt secret word, deterministic win).
Surface a winner ribbon in the verdict banner only when a winner exists, so
none-kind and legacy runs render byte-identically. Enforce the authoring
checklist in tests/test_scenario_contract.py and document the co…
feat: Implement well-known typed fields for verdicts in output models
agharsallah
ce159dc
- Introduced `_well_known_specs` function to define typed fields for `winner` and `scores` in agent output.
- Updated `build_output_model` to handle new field types, allowing `winner` to be optional and `scores` to default to an empty map.
- Enhanced `json_instruction` to provide specific hints for well-known fields in the output format.
- Modified `FishbowlSession` to derive `winner_kind` and `winning_models` based on verdicts, supporting team-based outcomes.
- Added validation tests for new competition configurations and verdict handling, ensuring proper behavior for both agent and team winners.
- Created comprehensive tests for the new verdict validation logic, including re-ask scenarios and score normalization.
feat: implement per-event model attribution for enhanced event tracking
agharsallah
a71301e
feat: implement Archive feature for past session management
agharsallah
a2ca0e0
- Added `archive.py` to handle listing and loading past runs for a specific user and scenario.
- Introduced `list_runs` function to retrieve past runs scoped by user session ID and scenario.
- Implemented `load_replay` function to create a read-only `ReplaySession` for a given run ID.
- Created helper functions for formatting run cards and timestamps.
- Enhanced CSS styles for the Archive drawer and run cards to improve UI presentation.
- Updated `FishbowlSession` and `ReplaySession` classes to support new functionality and ensure session isolation.
- Added comprehensive tests for the Archive feature, ensuring correct session attribution and replay functionality.
- Modified memory index tests to include run ID scoping for event searches.
- Created new tests for run history and lifecycle management, ensuring proper event emission and session handling.
feat: update agent personas and scheduling in YAML configurations; enhance event handling and memory deduplication in core logic
agharsallah
27b304e
feat: require live infrastructure for event store and inference, removing offline mode
New "Telemetry" tab in the Fishbowl app, reading live from the in-memory
telemetry store: a filterable structured-log feed (by level/layer), metric
charts (LLM calls, tokens in/out, agent-turn latency via gr.BarPlot/LinePlot),
and a per-trace timeline grouped by trace_id where each span reveals the actual
prompt + memory the agent saw. Manual Refresh + 3s auto-tick + filter-driven
repaint. New render/telemetry.py holds the pure store->view shaping.
Co-Authored-By: Codex <codex@openai.com>
Wire the observability facade through the load-bearing layers:
- models: llm.call/llm.structured spans with GenAI attrs, token/cost metrics,
full prompt+response captured at DEBUG (litellm, stub, openai-compat, router).
- conductor+governor: run/turn/agent.turn spans, run/turn/agent context binding,
event-append logs, agent-turn latency, governor budget-trip metrics+logs.
- memory: memory.recall + memory.index.search spans; DEBUG-logs the EXACT
salience-ranked memory text (with scores) each agent receives; index hit/latency
metrics and keyword-fallback logging; context.build structure logging.
- ledger/events/projections: per-append log+counter, projection.apply, invalid-kind.
Note: litellm_provider.py carries some pre-existing in-progress changes that ride
along with the instrumentation on this feature branch.
Co-Authored-By: Codex <codex@openai.com>
feat: add Hugging Face inference backend next to Modal
agharsallah
45917c7
Introduce a second, serverless inference backend (Hugging Face Inference
Providers) alongside the self-hosted Modal vLLM endpoints, and surface the
choice in the Fishbowl Lab.
- src/models/hf_catalogue.py: stdlib-only, offline-safe catalogue of small
instruct models (tiny<=4B .. strong<=32B); binding_for yields the same
LiteLLM custom-endpoint form, so the existing transport calls it unchanged.
- src/models/inference.py: one facade unifying Modal + HF behind
entries/binding_for/default_key_for_profile/backend_available. Models are
named by backend-qualified keys ("hf:<repo>"); a bare key means Modal, so
all existing keys/config/tests are untouched.
- router, registry endpoint: expansion, and has_live_credentials now read the
facade (live = any backend configured).
- Fishbowl Lab gains a headline "00 - Inference backend" radio (Modal vs HF)
driving the cast/judge pickers; the topbar chip names the live backend(s).
A composed run forces live on the chosen backend when it…
feat(tests): add tests for handling first-person openers in clean_clue function
agharsallah
a4ce257
feat: implement streaming agent responses and enhance conversation flow with repeat guards
agharsallah
0b0d9be
feat: enhance agent personas and improve model performance tuning
agharsallah
40a30b6
feat(core): add resilient loop and shared blackboard
agharsallah+1: Codex
ba6dd5f
Improve conversation stability and privacy across live scenarios by sharing spoken context, isolating per-agent failures, and hardening structured output parsing.
- Inject a shared WHAT'S BEEN SAID blackboard into agent context
- Continue turns when one agent fails while preserving budget-stop semantics
- Prevent secret leakage in scenario seed/memory rendering
- Harden JSON parsing and salvage logic for reasoning model outputs
- Tune model token budgets and diversify spy cast model tiers
- Add ADR-0023 plus regression tests for context, memory, parsing, and spy flow
Co-authored-by: Codex <codex@openai.com>
feat: Enhance model routing and deployment flexibility
agharsallah
c1656a8
- Introduced `vllm_version` to allow per-model inference stack pinning, enabling models to opt for nightly builds when necessary.
- Updated image building logic to handle model-specific vLLM versions, ensuring compatibility with various architectures.
- Added `model_endpoint` to `AgentManifest` for explicit model binding, allowing agents to route to specific models instead of relying solely on profiles.
- Enhanced `ModelRouter` to resolve specific catalogue models, improving routing accuracy and flexibility.
- Updated UI components in the Fishbowl Lab to support new model selection features, ensuring only deployable models are presented.
- Improved tests to validate new model selection and routing behaviors, ensuring robustness and adherence to expected functionality.
Merge branch 'feat/live-governor-reason' into feat/live-integration
agharsallah
d4eb3a3
fix: auto-load .env (Python) + normalize Neon postgres scheme for live mode
agharsallah+1: Codex
7eccfb0
The app read os.environ directly, so 'uv run app.py' ran offline unless the user
sourced .env first — and 'source .env' aborts on a Neon URL's '&' (and JSON values),
so DATABASE_URL/MEMORY_INDEX/MEM0_API_KEY never reached the process. Load .env in
Python at startup (real env wins via setdefault; empty values skipped; skipped under
pytest so the suite stays hermetically offline). Also normalize bare postgresql:// and
postgres:// to postgresql+psycopg:// so a copy-pasted Neon URL uses the shipped psycopg3
driver instead of failing on the absent psycopg2.
Co-Authored-By: Codex <codex@openai.com>
feat: add structured reason to BudgetExceeded + Governor.snapshot
agharsallah+1: Codex
ac3ed3b
Name which bound tripped (.reason) on BudgetExceeded and expose a read-only
Governor.snapshot, so the UI can show a precise stop message. Backward
compatible: same raise sites, thresholds, and BudgetExceeded(RuntimeError) type.
Co-Authored-By: Codex <codex@openai.com>
feat: add Fishbowl UI presenter layer and say-vs-think engine support
agharsallah+1: Codex
ade9df5
Implements the data foundation (Phases 0-1 of the Fishbowl UI plan) the
Gradio frontend will bind to — all additive, engine left untouched, modularity
preserved (the engine never imports the UI).
- src/ui/fishbowl/: derive_cast_state (per-agent {said,thought,mood} ledger
view), adapter (engine -> say/narrate/poke/verdict + hue/tier/voice/mood with
graceful fallbacks), view_model_at (pure prefix-replay snapshot at any scrubbed
step, with real tokens/rounds from the governor)
- DeterministicTinyModel is now schema-aware, gated on output_extra_fields: it
synthesizes thought/mood offline so the mind-reader works with no API key, while
plain agents stay byte-identical
- AgentManifest gains optional hue/archetype; Conductor.inject_user_event gains an
optional label; cast manifests declare output_extra_fields + presentation metadata
- tests/test_fishbowl.py (15 tests, incl. an offline proof the ledger carries
thought+mood); full suite 277 passed, ruff clean
- docs: manifest-spe…
feat: add optional mem0 cloud backend for memory index and update documentation
agharsallah
c3f5c19
feat: unify model catalogue and self-hosted routing
agharsallah+1: Codex
9dd6dab
- move model metadata to a shared modal/catalogue.py source of truth and keep modal/registry.py as a compatibility re-export
- add src/models/modal_catalogue.py bridge and resolve config/models.yaml endpoint keys into concrete model/base_url/api_key bindings
- make live-path detection depend on Modal bindings (MODAL_WORKSPACE or MODAL_LLM_BASE_URL) and remove generic cloud-model fallback defaults
- switch memory index default embeddings to local sentence-transformers and document the off-grid behavior
- add ADR-0019 and update architecture/deployment docs, env examples, tests, and lockfile
Co-authored-by: Codex <codex@openai.com>
feat: mem0 as a layered, derived semantic memory index
agharsallah+1: Codex
f487b74
Add a MemoryIndex abstraction and a mem0-backed implementation that upgrades the
salience relevance term with semantic search. Layered + derived: the index is
populated from ledger events (keyed by event.id, idempotent and rebuildable), so
the append-only ledger remains the single source of truth (ADR-0005 preserved).
Env-gated by MEMORY_INDEX; offline keeps the keyword path. 258 passed, 2 skipped.
ADR-0018.
Co-Authored-By: Codex <codex@openai.com>
feat: Instructor-validated structured output on the live path
agharsallah+1: Codex
99293d7
Replace prompt-and-regex JSON with Instructor (Pydantic-validated, retried) on
the live path, wrapping the LiteLLM client; the kind is constrained to may_emit
by construction, so malformed prose can no longer enter the ledger. Degrade
chain: structured -> tolerant parser -> _raw_fallback (offline stub keeps the
parser). Token/cost metering preserved. 226 passed, 1 skipped. ADR-0016.
Co-Authored-By: Codex <codex@openai.com>
feat: LiteLLM gateway routing profiles to Modal-served small models
agharsallah+1: Codex
7df0a45
Add LiteLLMProvider; ModelRouter binds each profile (tiny/fast/balanced/strong)
to its Modal vLLM endpoint (workspace env-templated, not hardcoded) and meters
real token + cost usage into the Governor, making hourly_budget_usd live. Env-
gated; offline keeps the deterministic stub. 212 passed, 1 skipped. ADR-0015.
Co-Authored-By: Codex <codex@openai.com>
feat: env-gated Postgres event store via SQLAlchemy (layered)
agharsallah+1: Codex
98938c3
Add SqlAlchemyLedger — a durable backend behind the Ledger interface, driving
both Postgres (Neon) and SQLite through one code path. The append-only ledger
stays the source of truth; backend selection is env-gated by DATABASE_URL with
in-memory fallback, so the suite stays green offline (201 passed, 1 Postgres-
gated skip). eventsourcing evaluated and declined (flat-envelope events don't
map to its aggregate model) — rationale in ADR-0014.
Co-Authored-By: Codex <codex@openai.com>
Refactor code structure for improved readability and maintainability
Five new engine modules (zero breaking changes to existing tests):
Manifest system (src/core/manifest.py):
- AgentManifest: full declarative spec — identity, communication contract,
scheduling, model profile, memory config, capability grants
- ModelProfile: tiny|fast|balanced|strong → resolved to concrete model name
via env vars (MODEL_TINY / MODEL_FAST / MODEL_BALANCED / MODEL_STRONG)
- MemoryConfig: window, use_salience, salience_top_k, reflection_threshold
- ScheduleConfig: tick_every, max_consecutive
Structured output (src/core/structured.py):
- json_instruction(): appends JSON constraint block to every agent prompt
- parse_agent_output(): three-tier parser (clean JSON → embedded JSON → fallback wrap)
- _raw_fallback flag for compliance rate tracking
- Works with any model on any endpoint — constraint is in the prompt, not tool-use
Salience-scored memory (src/core/memory.py):
- SalienceMemory: ranks visible events by relevance×recency×importance score
…
Engine additions (zero scenario edits required):
- EpisodicMemory: per-agent filtered view over the ledger; agents see only their
own actions and globally-visible event kinds, capped at 8 events for small-model
context windows
- ContextBuilder: single point of prompt assembly (persona → scene → memory →
visitor disturbances); changing prompt strategy is now a one-file edit
- Governor: enforces max_turns / max_calls_per_turn / max_total_calls; raises
BudgetExceeded before a runaway cascade can burn quota
Model provider:
- OpenAICompatProvider: works with any OpenAI-compatible endpoint; config via
OPENAI_API_KEY / OPENAI_BASE_URL / MODEL_NAME; falls back to deterministic stub
when no key is present
Agents + scenarios:
- All four Tiny Wood agents use ContextBuilder with rich personas and episodic memory
- New EchoAgent transforms visitor injections through the wood's logic
- Mystery Roots (src/scenarios/mystery_roots.py): convergent blackboard swarm with
ClueGatherer / Hyp…
