diff --git a/SCENARIO_UNIQUENESS_AUDIT.md b/SCENARIO_UNIQUENESS_AUDIT.md new file mode 100644 index 0000000000000000000000000000000000000000..9e5e6c0cdb39adb6c223c36f255c514212742e5e --- /dev/null +++ b/SCENARIO_UNIQUENESS_AUDIT.md @@ -0,0 +1,496 @@ +# Scenario Uniqueness Audit (Phase 2) + +Read-only audit of `openra_bench/scenarios/packs/*.yaml`. Scope: 212 YAML +files (excluding `TEMPLATE.yaml`); 23 in-flight `econ-*` packs were +deliberately skipped (other agents own them), leaving **189 packs probed**. + +The probe was a stall policy — `Command.observe()` only — run on the +hard tier, seed=1, of every pack, through the same code path +(`openra_bench.eval_core.run_level`) the bench evaluator uses. Each +result is the engine-evaluated outcome (`win` / `loss` / `draw`), +final game tick, and the static profile read straight from the YAML. + +Headline numbers +- 181 / 189 packs (95.8%) — stall LOSSES (healthy: no-cheat bar holds). +- 3 packs — stall WINS (the no-cheat bar is broken). +- 3 packs — stall DRAWS (no real LOSS reachable from a stall play). +- 2 packs — Rust engine panic at `reset(seed=1)` (both + `adversarial-siege` / `adversarial-skirmish` — already + `status: quarantine` in-pack so they are NOT in the default eval set, + but they remain on disk and panic when explicitly invoked). + +Quarantined packs are still YAML-discoverable and were probed. They are +flagged `[Q]` in the lists below. + +--- + +## Section 1 — Defect packs + +A "defect" here means stall WINS (the lazy bar fell) or stall DRAWS +(no real reachable LOSS — the timeout collapses to draw degeneracy +because the fail predicate never trips). Both are explicit violations +of the "no defect, no cheat" bar in `CLAUDE.md`. + +### 1A. Stall WINS (3 — bar fell) + +| Pack | Outcome | Turn | Tick | Diagnosis | +|---|---|---|---|---| +| `mid-economy-under-fire` | **WIN** | 11 | 993 | Stall achieves the win predicate without any agent action. Hard win-clause is `economy_value_gte:4000 AND harv≥2 AND units_lost_lte:2`. With the current engine, the 3 starter harvesters auto-harvest (no `harvest` command needed — they begin in `harvest` mode), the perimeter 1tnks auto-fire and kill the lone raider 1tnk, and EV trips 4000 by tick ~993 with zero losses. The pack header says "Stall (only observe)… harvs never harvest → EV stays at 0 → LOSS." That is no longer true under the current engine — harvs in `harvest` mode work without re-issued orders. **Fix**: either remove the harvesters from the starting placement (require the model to issue `Command.harvest`) or replace the `1tnk` raider with a stronger raider wave that out-attritions an idle defense ring. | +| `combat-naval-shore-strike` | **WIN** | 3 | 198 | Stall wins inside 3 turns. The hard tier places two destroyers in the water channel on `stance:2` (Defend), and the entire shore garrison on `stance:0` (HoldFire). The destroyers auto-fire on every garrison unit in range and clear all 5 enemies in <200 ticks; the agent never sends a command. **Fix**: set the destroyers to `stance:0` so the agent must issue an `attack_unit` order; or place the shore garrison just outside the destroyer's auto-target range so the agent must manually fire. The whole capability (target an across-shore target with a ranged unit) collapses because the engine fires automatically. | +| `def-with-ambush` | (intended) **WIN** | 24 | 1233 | **Not a defect — exempt by design**. CLAUDE.md `§Triage coverage`: "1 pack (`def-with-ambush`) is exempt by design (positional-discipline scenario where do-nothing IS the intended policy)." The capability under test is *hold the ambush position*; the four `stance:2` flanker tanks auto-engage the rusher band when it enters weapon range. Listed here only for completeness — no action required. | + +### 1B. Stall DRAWS (3 — no LOSS reachable) + +A draw means the win predicate was never met AND the fail predicate +never tripped, so the level resolves to `draw` (outcome score 0.5). +This is a defect because it makes the lazy play indistinguishable +from a partial / borderline play, and inflates the lazy score by +treating it as half a win. + +| Pack | Outcome | Turn | Tick | Diagnosis | +|---|---|---|---|---| +| `economy-time-box` [Q] | **DRAW** | 80 | 7203 | Already quarantined in-file (reason: "redundant with economy-force-buildup"). The hard tier has **no `fail_condition` at all** (`fail_condition: None`), so a 12000-tick budget that never produces 6 units / 5 buildings simply times out as draw. This is one of the canonical CLAUDE.md defect classes ("no `fail_condition`, or only triggers on full force-wipe; a stall / preserve / partial outcome silently draws"). Since the pack is quarantined the lazy-bar drift is harmless to the default eval; if the pack is ever un-quarantined, add `fail_condition: {after_ticks: 12001}` plus `{not: own_units_gte: 1}`. | +| `spec-spy-infiltrate` | **DRAW** | 3 | 273 | Hard tier `fail_condition: {after_ticks: 4001}` — i.e. the only failure mode is the deadline. But the agent's only combat-relevant units are 2 spies (`spy`) and the scenario also places one defender `e1`. With stall, the spies are passive while the engine auto-`done`s at turn 3 (likely because the defender `e1` falls to auto-fire or because all agent combat units are destroyed). The outcome lands as DRAW because the spy-only force is wiped before the within_ticks ever bites. **Fix**: add `{not: own_units_gte: 1}` (or `unit_type_count_gte:{spy, 1}`) to the `fail_condition.any_of` so a spy-wipe is a LOSS, not a draw. Same fix likely needed for `easy` / `medium` (uninspected; consider auditing). | +| `def-bridge-chokepoint` | (mixed: usually **LOSS** at tick 1623, occasionally **DRAW** in one fresh process) | 18 | 1623 | The first probe (subprocess #1) reported DRAW; three subsequent re-runs from a fresh process consistently reported LOSS (17 losses, fail-clause `not has_building:fact` trips when the procs/fact fall under the rifle assault). This is borderline — the pack is *probably* healthy, but the one-off DRAW observation suggests a possible determinism-edge case (race between the engine auto-`done` and the predicate evaluator). Recommend a 4-seed re-probe (1–4) over a clean Python process to confirm. If the LOSS is stable, no fix required; if the DRAW resurfaces, add `{not: own_units_gte: 1}` to the fail clause. | + +### 1C. Engine panics at reset (2) + +| Pack | Status | Error | +|---|---|---| +| `adversarial-siege` | `quarantine` (consolidated into `adversarial-duel`) | `pathfinder.rs:175 index out of bounds: len 5120, index 5382` at `env.reset(seed=1)`. The hard tier places actors outside the playable bounds (CLAUDE.md footgun #6) for the chosen spawn. Pack is excluded from default eval; no eval-time impact, but flag the file is still selectable via explicit `--packs`. | +| `adversarial-skirmish` | `quarantine` (consolidated into `adversarial-duel`) | Same panic at `env.reset`. Same situation as above. | + +Recommend either deleting these two YAMLs (the consolidation is the +documented future state) or moving them to `packs/_archived/` so +`discover_packs` skips them (it already skips `_*` and `TEMPLATE`). + +--- + +## Section 2 — Duplicate / near-duplicate clusters + +Methodology: each pack was fingerprinted by +`(capability, tools, actor_types_sorted, win_predicate_keys_sorted)`. +Five strict (= all four fields equal) clusters surfaced; a second pass +relaxed `tools` and `actor_types` to surface near-duplicates with the +same capability + same win-key set, which surfaced a further ~10 +clusters. Each cluster is judged on the per-pack `real_world_meaning` +and the `level_description` of the hard tier — if these tell the same +story, the cluster is genuinely redundant; if not, the predicate-key +match is coincidental and the packs probe distinct skills. + +### 2A. Strict duplicates (action required) + +1. **`adversarial-siege` + `adversarial-skirmish`** — both already + quarantined as "consolidated into `adversarial-duel`". **Keep:** + `adversarial-duel`. **Action:** physical removal (delete or + `_archive`) — they are the only two packs that crash the engine at + load, so removing them eliminates the crash class entirely. + +2. **`artofwar-decoy-sacrifice` + `artofwar-lure-the-tiger`** — both + `capability: reasoning`, both use bot `guard`, identical hard-tier + actor list `[2tnk,e1,e3,fact,jeep]`, identical win-keys + `{units_in_region_gte, units_lost_lte, within_ticks}`, both + described as "main-force reaches an objective by diverting a + leashed defender." **Keep:** `artofwar-lure-the-tiger` (the more + doctrinally complete framing — leash mechanic; "the strong + defender" is the load-bearing pull). **Merge / delete:** + `artofwar-decoy-sacrifice` — its "spend a decoy unit" angle is + covered by the `units_lost_lte` clause in the lure pack, which + already permits a small attrition budget the model can spend on + the bait sub-force. + +3. **`scout-cycle-keep-info-fresh` + `scout-track-enemy-movement`** — + both `capability: perception`, identical hard-tier actor list, both + use the `then`-chained `units_killed_gte` win predicate, both + probe "scout multiple times because the world changes between + observations." Both use the `scheduled_events:` hook (cycle = + reinforcement waves; track = enemy march legs). **These are + genuinely distinct skills** — cycle is "re-observe a stale region + for new arrivals", track is "follow a moving target across the + map." **Keep both**, but verify the briefing of each clearly + distinguishes the cause of staleness so the model is not asked to + solve cycle-by-track-strategy or vice-versa. (No action required if + the briefings are unambiguous, which they appear to be on + inspection.) + +4. **`build-defensive-skirt-corners` + `build-defensive-tower-line`** + (strict 4-field match) **AND** the wider quartet + `build-defensive-tower-cluster` + `def-in-depth-vs-single` + (same capability + win-key set, near-identical actor list, all use + `rusher` bot). These four packs all probe "where do you place a + finite pillbox budget to survive an incoming rush?" with topology + variants: skirt (one in each map-relative corner of the building), + cluster (tight wrap), line (across the choke), depth (two thinner + bands). **Each topology is genuinely distinct in the optimal + answer**, so all four packs probe a real choice axis. **Keep all + four** but consider promoting them into a single named "Defense + Topology Suite" in the eval-cell catalog so the four cells score + together (one "topology-IQ" metric) rather than as four unrelated + reasoning packs. (No file action required; this is a catalog / + metric grouping recommendation.) + +### 2B. Near-duplicates by win-predicate keys (informational — same +predicate idiom is used across distinct scenarios; verify each is +load-bearing) + +These groups all share an identical `(capability, win_predicate_keys)` +fingerprint. Each group below is **not necessarily a duplicate**, but +flags that all members rely on the same predicate idiom and the +scenarios should diverge on actor composition / pressure / bot to +remain discriminative. + +- `(reasoning, then+within_ticks)` — N=10: + `build-power-online-first`, `build-sequence-tech-cheapest`, + `build-sequence-tech-fastest`, `build-sequence-tech-most-resilient` + (not in cluster — was filtered earlier), `lh-econ-army-victory`, + `lh-opening-to-defense-to-counter`, `lh-opening-to-tech-to-army`, + `lh-progression-stage-locked`, `rob-objective-change-midway`, + `tech-balanced-econ-then-tech`, + `lh-build-army-coordinate-multifront-attack`. These are all + ordered-multi-phase scenarios. Distinct intents (build-order vs + long-horizon vs objective-shift); the `then` predicate is the + common engine machinery, not a duplication signal. + +- `(reasoning, building_count_gte+building_in_region+within_ticks)` — + N=7: `build-sell-and-rebuild-elsewhere`, `def-in-depth`, + `def-tower-line-vs-cluster`, `expansion-aggro-3-base-greedy`, + `mcv-deploy-second-base`, `mcv-deploy-third-base`, + `mfb-mirror-base-east-west`. All probe "build / re-build N + buildings in a target region." Genuinely distinct intents + (expansion-greed, mirror-base symmetry, MCV-redeploy under + pressure); keep all. + +- `(reasoning, building_count_gte+units_killed_gte+within_ticks)` — + N=6: `adv-rps-counter-pick`, `build-tech-skip-decision`, + `def-counter-battery`, `def-walls-vs-towers`, `def-while-building`, + `lh-tech-rush-vs-army-rush`. All "kill K enemies and have N + buildings standing." Reads as honest differentiated tests. + +- `(action, own_units_gte+units_killed_gte+within_ticks)` — N=5: + `combat-flanking-attack`, `combat-harass-aggro-commit`, + `combat-kite-and-pull`, `combat-kite-jeep-vs-tank`, + `combat-tanya-vs-rush`. Five combat micro packs — kite, flank, + harass. Each probes a different micro idiom; keep all but verify + that the win bar (`units_killed_gte`) is tuned per pack so the + intended micro is load-bearing, not just "auto-fire wins." + Two are explicitly kite (`combat-kite-and-pull`, + `combat-kite-jeep-vs-tank`) — **possible duplicate**: kite-and-pull + uses generic 2tnk/3tnk vs jeep-vs-tank's unit-class asymmetry. + Keep both only if `combat-kite-jeep-vs-tank` actually requires the + jeep speed advantage that `combat-kite-and-pull` doesn't (read the + briefings; they look distinct). + +- `(action, reach_region+units_lost_lte+within_ticks)` — N=4: + `proc-no-attack-passive-only`, `proc-tool-use-multi-distractor`, + `proc-tool-use-with-distractor`, + `strict-toolban-fidelity-under-pressure`. All + procedural-compliance packs — "reach the goal without using a + forbidden tool / without attacking." The `proc-tool-use-*` pair is + a possible duplicate: "with-distractor" (one distractor tool) vs + "multi-distractor" (multiple). Both are the same skill at different + scales — recommend collapsing into a single pack with three tiers + (no distractor / one / many) rather than two top-level packs. + +- `(perception, buildings_discovered_gte+units_lost_lte+within_ticks)` + — N=4: `perception-target-vs-fog`, `scout-detect-enemy-tech`, + `scout-discover-hidden-base`, `scout-multiple-fog-areas`. All + "discover K enemy buildings under fog." Each probes a distinct + discrimination (target-vs-fog = ignore decoys; detect-enemy-tech = + read the tech-tree; hidden-base = single hidden compound; + multiple-fog-areas = K disjoint regions). Keep all. + +### 2C. Overlapping prefixes worth a catalog review (not duplicates, +but the area is dense) + +- `combat-*` (25 packs), `def-*` (18), `scout-*` (14), `build-*` (14), + `lh-*` (13), `proc-*` (10), `mfb-*` (8), `rob-*` (8), `tp-*` (7), + `coord-*` (7), `mcv-*` (6), `spec-*` (5), `economy-*` (5), + `tech-*` (4), `strategy-*` (4), `perception-*` (4), `artofwar-*` (4), + `tempo-*` (2), `maint-*` (2), `mid-*` (2), `expansion-*` (3), + `coordination-*` (2), `harass-*` (1), `risk-*` (1), `power-*` (1), + `navigation-*` (1), `defense-*` (1), `longhorizon-*` (1), + `custom-*` (1), `building-*` (1), `rush-*` (1), `reasoning-*` (2), + `action-*` (2), `adv-*` (2), `adversarial-*` (3), `strict-*` (3). + The combat / def / scout / build axes carry ~70 packs combined; a + follow-up audit explicitly checking pairwise overlap inside each + prefix (briefing-level read) would surface ~5–10 more candidate + collapses. + +--- + +## Section 3 — Capability coverage matrix + +Cross-referenced against the **phase × decision-type** matrix sketched +in `PAPER_PLAN.md §12.2` and `CLAUDE.md`. Each cell lists the packs +that primarily measure it (a pack may legitimately serve more than one +cell — only the dominant assignment is listed; cross-cutting packs are +flagged). + +### Opening + +| Decision | Packs | Density | +|---|---|---| +| MCV deploy site (where to plant) | `mcv-deploy-and-build`, `mcv-deploy-defensible-site`, `mcv-deploy-near-resource`, `mcv-deploy-relocate-under-pressure`, `mcv-deploy-second-base`, `mcv-deploy-third-base` | **6 packs** (dense; good) | +| Build-order commit | `build-sequence-tech-cheapest`, `build-sequence-tech-fastest`, `build-sequence-tech-most-resilient`, `build-tech-skip-decision`, `tech-balanced-econ-then-tech`, `tech-aggro-all-in`, `tech-turtle-defensive-tech`, `tech-production-planning`, `build-power-online-first`, `power-budget-online`, `building-and-planning` | **11 packs** (dense; good) | +| Defense-direction commit (which side to anticipate) | `def-position-expected-direction`, `def-position-revealed-direction`, `def-multi-direction`, `def-surprise-flank-react`, `def-pre-position-mobile-reserve` | **5 packs** (good) | +| Rush-defense | `defense-rush-survive`, `rush-hour`, `build-defensive-tower-cluster`, `build-defensive-tower-line`, `build-defensive-skirt-corners`, `def-in-depth`, `def-in-depth-vs-single`, `def-walls-vs-towers`, `def-tower-line-vs-cluster`, `def-while-building` | **10 packs** (dense) | + +### Early-mid + +| Decision | Packs | Density | +|---|---|---| +| Harass / harass-preserve | `harass-response-preserve`, `combat-harass-aggro-commit`, `combat-harass-balanced-hit-and-run`, `combat-skirmish-then-disengage`, `combat-bait-counter-attack`, `combat-kite-and-pull`, `combat-kite-jeep-vs-tank`, `combat-retreat-after-engagement`, `combat-suicide-charge-mission` | **9 packs** (dense) | +| Exact-count perception | `perception-count-the-threat`, `perception-count-the-threat-small-k`, `scout-count-defenders`, `scout-detect-incoming-army` | **4 packs** (adequate) | +| Scout-direction commit | `scout-detect-base-direction`, `scout-far-frontier`, `scout-frontier-reading` (=`perception-frontier-reading`), `scout-multiple-fog-areas`, `scout-and-report`, `scout-and-survive`, `scout-discover-hidden-base`, `scout-detect-enemy-tech`, `scout-map-reveal-percent-target`, `reasoning-frontier-commit` | **10 packs** (dense; possible over-coverage) | + +### Mid + +| Decision | Packs | Density | +|---|---|---| +| Live-economy defense | `mid-economy-under-fire` (**DEFECT — stall wins**), `econ-harvester-defense-raid` [skipped — econ-*], `econ-protect-harvester-route` [skipped] | **~3 packs, 1 defective** — the only non-econ exemplar is broken. **Gap when the in-flight econ-* land — verify they cover.** | +| Tech-switch on scout | `mid-tech-switch-on-scout`, `lh-scout-react-counter`, `adv-rps-counter-pick` | **3 packs** (adequate) | +| Second front | `mfb-base-1-defend-base-2-build`, `mfb-supply-line-link-between-bases`, `mfb-mirror-base-east-west`, `mfb-two-base-simultaneous`, `mfb-third-base-against-clock`, `expansion-balanced-2-base-defended`, `expansion-aggro-3-base-greedy`, `coord-diversionary-attack` | **8 packs** (dense) | +| Replan after loss | `rob-unit-loss-recovery`, `rob-partial-base-loss-continue`, `lh-recovery-after-mid-game-loss`, `build-engineer-rebuild-after-loss`, `def-retreat-and-rebuild`, `build-sell-and-rebuild-elsewhere` | **6 packs** (good) | + +### Mid-late + +| Decision | Packs | Density | +|---|---|---| +| Concede vs hold | `mid-concede-vs-hold` | **1 pack** (THIN — single exemplar) | +| Isolate vs split | `combat-divide-and-conquer`, `combat-pincer-coordination`, `combat-prevent-retreat`, `combat-formation-tank-wedge` | **4 packs** (adequate) | +| Tempo double-window | `tempo-double-window`, `tempo-strike-window`, `coordination-staggered-window`, `tp-survive-and-strike-at-window` | **4 packs** (adequate) | +| Decoy / lure / feint | `artofwar-decoy-sacrifice`, `artofwar-lure-the-tiger`, `artofwar-indirect-approach`, `artofwar-sequenced-citadel`, `combat-bait-counter-attack`, `coord-diversionary-attack`, `def-with-ambush` | **7 packs** (good) | +| Multi-front | `mfb-rotating-production-pressure`, `mfb-redundant-tech-buildings`, `mfb-tech-base-vs-economy-base`, `rob-multiple-simultaneous-pressures` | **4 packs** (adequate) | + +### Late + +| Decision | Packs | Density | +|---|---|---| +| Sustained multi-front | `lh-build-army-coordinate-multifront-attack`, `mfb-rotating-production-pressure` | **2 packs** (thin) | +| Base-trade race | None directly | **0 packs (GAP)** | +| Counter-strategy read | `adv-rps-counter-pick`, `combat-attack-from-behind-fog`, `lh-tech-rush-vs-army-rush` | **3 packs** (adequate) | +| Superweapon timing | `spec-nuke-strike`, `spec-tanya-c4-strike`, `spec-engineer-capture`, `spec-spy-infiltrate` (**DEFECT — stall draws**), `spec-thief-steal-cash` | **5 packs, 1 defective** (good once spec-spy is fixed) | +| Credit-only final phase | `lh-credit-only-final-phase` | **1 pack** (thin) | + +### Cross-cutting + +| Capability | Packs | Density | +|---|---|---| +| Procedural compliance under pressure | `proc-checklist-no-deviation`, `proc-conditional-branch-action`, `proc-instruction-following-edge-case`, `proc-no-attack-passive-only`, `proc-only-build-no-combat`, `proc-only-defend-no-attack`, `proc-ordered-action-strict`, `proc-strict-toolban-fidelity`, `proc-tool-use-multi-distractor`, `proc-tool-use-with-distractor`, `strict-production-bom`, `strict-sequence`, `strict-toolban-fidelity-under-pressure`, `tp-pressure-procedural` | **14 packs** (very dense) | +| Long-horizon multi-phase | `lh-*` (13 packs), `longhorizon-opening-to-assault`, `lh-100-turn-marathon-survival` | **14 packs** (dense) | +| Coordination across squads | `coord-converge-on-target`, `coord-cover-and-move`, `coord-diversionary-attack`, `coord-mutual-support`, `coord-relay-attack`, `coord-relay-vision-chain`, `coord-squad-handoff`, `coordination-ordered-rendezvous`, `coordination-staggered-window`, `action-multiunit-coordination`, `action-sequenced-execution`, `combat-pincer-coordination`, `combat-heli-flank` | **13 packs** (dense) | +| Adversarial 1v1 (full macro) | `adversarial-duel`. The full 1v1 battleground lives in `one_v_one.py` (`openra_bench/`), not as a `meta.capability: adversarial` pack. | **1 pack** (the catalog says this is by design — full macro is the live ladder) | + +--- + +## Section 4 — Capability gaps (uncovered or thin cells) + +Cells worth new packs, in priority order: + +1. **Base-trade race (mid-late / late)** — *no* current pack tests + "your base falls while you race the enemy's; whoever finishes first + wins." Sketch: hard tier places both players' bases halfway to + killable, no defenders left; the agent has one army worth attacking + with, the enemy is doing the same to the agent's base; win by + destroying the enemy's `fact` before they destroy yours. Win + `building_count_gte:{type:fact,n:0,owner:enemy}` paired with + `has_building:fact` (own) and an aggressive `within_ticks`. Pack id: + `combat-base-trade-race`. + +2. **Concede-vs-hold (mid-late)** — only `mid-concede-vs-hold` + currently. Sketch a second pack: `mid-concede-vs-rebuild` where a + forward outpost is lost-cost (committing reinforcements throws + good money after bad), and the right call is to abandon it and + rebuild on a held line. Win condition: keep the rear-line `fact` + alive AND have ≥4 `tent`/`weap` on the rear line at deadline; + throwing reinforcements at the forward outpost fails the resource + budget. Pack id: `mid-concede-forward-rebuild-rear`. + +3. **Live economy defense (mid)** — the only non-econ pack + (`mid-economy-under-fire`) is currently a stall-WIN defect. After + fixing it, add a sibling pack `mid-economy-rebuild-harvester-line` + where the agent must re-issue `harvest` commands to a freshly + produced harvester after a raider kills the starter ones (puts + the load-bearing capability on the `harvest` order, not on the + passive auto-harvest). + +4. **Long-horizon credit-only final phase** — `lh-credit-only-final-phase` + is the only exemplar. Sketch sibling: `lh-credit-only-bait-window` — + the agent must spend the last credits on a single decisive strike + window rather than on attrition. Different decision (one shot vs + accumulation). + +5. **Sustained multi-front (late)** — thin (2 packs). Sketch: + `mfb-three-front-rotation` — three simultaneously-attacked bases + with the agent's army too small to defend all three at once; the + decision is which two to hold and which one to let fall while the + army cycles. Cross-pack with `coord-relay-attack`. + +6. **MCV deploy under timer + crossfire** — current MCV packs cover + site selection but not "deploy NOW or the MCV is destroyed by an + incoming wave that hits in 4 turns." Sketch: + `mcv-deploy-emergency-relocation` — the starting MCV stands in the + path of an incoming squad; the agent must deploy + re-build OR + move + re-deploy further west before the squad arrives. + +7. **Information freshness across modalities** — `scout-cycle-keep-info-fresh` + and `scout-track-enemy-movement` cover this for the structured + channel, but the perception ablation grid (channel × fog) doesn't + currently have an information-freshness pack that is explicitly + easier to solve with the labelled image (the `image` channel is + advantaged when the model can spot newly-spawned units + inter-tick). Sketch: `perception-freshness-image-advantage`. + +8. **Single-pack `meta.capability: adversarial`** — only + `adversarial-duel` carries this tag. PAPER_PLAN.md §12.2 notes the + imbalance. The full 1v1 lives in `one_v_one.py` (correct), but a + second adversarial-tagged pack that tests "reactive opponent + selects a counter from a small menu mid-game" (different from RPS + pre-game commit) would put real teeth on the tag. + +9. **Engineer / Tanya / spy under fire** (specialist packs) — only one + pack each (`spec-engineer-capture`, `spec-tanya-c4-strike`, + `spec-spy-infiltrate`, `spec-thief-steal-cash`, `combat-tanya-vs-rush`). + Each specialist has one happy-path scenario. Sketch a "stealth + + target priority" pack per specialist that requires the model to + pick *which* of N enemy assets to hit with the one-shot specialist. + +10. **Naval / amphibious** — `combat-naval-shore-strike` is the only + naval pack (and it's currently a stall-WIN defect). Once fixed, + add `combat-naval-amphibious-landing` (a transport ship lands + infantry on a contested shore) and `combat-naval-anti-air-vs-bomber` + pending the air-unit engine work. These are documented as + out-of-scope in `PAPER_PLAN.md §11.1` for the air variant but the + naval ones are now feasible thanks to `water_rect:`. + +--- + +## Section 5 — Map mis-bindings (rush-hour-arena used where intent +calls for a custom map) + +Of 189 probed packs, **183** use `rush-hour-arena`; **3** use a custom +map (`navigation-confined-hard-only`, `custom-map-no-enemy`, +`strategy-dilemma`/`-gauntlet`/`-twobody`); **1** uses a generator +(`combat-naval-shore-strike`); **2** are quarantined-and-crashing. + +Packs whose stated intent calls for a non-rush-hour geometry: + +| Pack | Stated intent | Current map | Recommendation | +|---|---|---|---| +| `combat-heli-flank` | Helicopter assault from a flank; the brief explicitly references aircraft flight over terrain. | rush-hour-arena (no terrain features to flank around) | Author / use a custom map with a forested ridge that channels ground units one way and aircraft another. (Engine air-unit support is out-of-scope per PAPER_PLAN §11.1 — pack is currently a ground-only proxy.) | +| `def-bridge-chokepoint` | "A water band cuts the map east-to-west with bridges." | rush-hour-arena + `water_rect` overlay | The `water_rect` overlay correctly synthesizes the bridge geometry on top of rush-hour, but the chokepoint-narrowing aspect ("attackers must pass through 3 narrow openings") works against an open-arena map. Consider a custom `bridges-arena` map authored once — the visual / minimap reading test is more honest when the bridges are real terrain rather than a YAML overlay on an unrelated map. | +| `combat-naval-shore-strike` | Naval ship in a water channel. | uses `naval-arena` generator (correct) — but the generator spec lives in the pack file rather than a named `.oramap`. | When a second naval pack is authored (per gap #10), promote `naval-arena` to a real `data/maps/naval-arena.oramap` so the bench has a canonical naval geometry. | +| `def-with-ambush` | "Concealed flanking defenders catch the band in an L-ambush down a lane toward the construction yard." Doctrinal answer requires real linear terrain. | rush-hour-arena (open) | The pack synthesizes the lane purely with actor placement (a single `e1` fixing defender at x=15, flankers at x=40). Visually the geometry is invisible on the minimap — a custom `corridor-arena` map with a real lane would make the spatial decision legible from the image. (Capability is sound; this is a perception-channel polish issue.) | +| `mfb-supply-line-link-between-bases` | "Supply line corridor between two bases." | rush-hour-arena | Similar polish issue — a custom multi-base map with two separated valleys would make the supply-line decision visible on the minimap rather than implicit in the actor coordinates. | +| `combat-hold-chokepoint` | "Defend a narrow chokepoint corridor." | rush-hour-arena (no narrow corridor) | Same; either author a `chokepoint-arena` map or accept that the chokepoint is implied by enemy spawn geometry alone. | +| `mcv-deploy-second-base`, `mcv-deploy-third-base` | "Plant a second / third base at a defensible site." | rush-hour-arena (uniform; no defensible sites) | The map is symmetric and uniform — every cell is roughly as defensible as every other. The capability becomes "pick a cell that satisfies the `building_in_region` predicate," not "pick a *defensible* cell." Add a custom map with terrain features (cliff edges, narrow passes) so the defensibility geometry is real. | +| `expansion-aggro-3-base-greedy`, `expansion-balanced-2-base-defended`, `expansion-turtle-1-base-fortified` | Expansion decisions over distinct base sites. | rush-hour-arena (only ~3 sensible base sites given map size) | Authoring a wider arena with 5+ distinguishable expansion sites would put real teeth on the trilemma. | + +**Summary**: roughly 8–10 packs would benefit from a custom map, but +all are currently *functional* (the no-cheat bar holds). The map +mis-binding is a perception-channel polish issue, not a correctness +issue. **Priority**: author 2 custom maps that unlock several packs +each: +1. `bridges-arena` — would replace `water_rect` overlay on + `def-bridge-chokepoint` and could host `combat-naval-amphibious-landing`. +2. `chokepoint-arena` — narrow corridor for `combat-hold-chokepoint`, + `def-with-ambush`, `mfb-supply-line-link-between-bases`. + +--- + +## Section 6 — Recommended actions (prioritized) + +### P0 — Hard defects (gates the no-cheat headline number) + +1. **Fix `mid-economy-under-fire`** — currently the only stall-WIN + defect among non-exempt packs. Either: + - (a) Remove the 3 starter harvesters; require the model to issue + `Command.harvest(...)` on a fresh harvester (the load-bearing + capability), OR + - (b) Upgrade the raider to a force that out-attritions an idle + defense ring (e.g. 3× `1tnk` + 2× `e3` over 60 turns), so a + stall play loses harvesters and the `harv,2` clause fails. + +2. **Fix `combat-naval-shore-strike`** — set the destroyer `stance` to + `0` (HoldFire) so the agent must issue an explicit `attack_unit` + order. Verify the intended capability (manual cross-shore attack) + still wins; stall now loses by deadline. + +3. **Fix `spec-spy-infiltrate`** — add `{not: own_units_gte: 1}` (or + `{not: unit_type_count_gte: {type: spy, n: 1}}`) to the + `fail_condition.any_of` so a spy-wipe is a real LOSS, not a + DRAW. Audit the easy / medium tiers for the same defect. + +4. **Delete or `_archive/` `adversarial-siege` and `adversarial-skirmish`** — + both already quarantined; deletion removes the only two + load-time-crashing packs. + +5. **Re-probe `def-bridge-chokepoint` over seeds 1–4 in clean processes** + — confirm the one-off DRAW observation was a transient (subsequent + re-runs gave LOSS consistently); if the DRAW recurs, add + `{not: own_units_gte: 1}` to its `fail_condition.any_of`. + +### P1 — Coverage gaps (new packs to author) + +In priority order from §4: `combat-base-trade-race`, +`mid-concede-forward-rebuild-rear`, `mid-economy-rebuild-harvester-line`, +`mfb-three-front-rotation`, `mcv-deploy-emergency-relocation`, +`lh-credit-only-bait-window`, `perception-freshness-image-advantage`, +`combat-naval-amphibious-landing`, and per-specialist target-priority +packs (`spec-engineer-priority-targets`, `spec-tanya-priority-targets`). + +### P2 — Duplicate consolidation + +1. Merge / delete `artofwar-decoy-sacrifice` into + `artofwar-lure-the-tiger`. +2. Merge `proc-tool-use-with-distractor` + `proc-tool-use-multi-distractor` + into a single pack with two tiers. +3. Re-read `combat-kite-and-pull` vs `combat-kite-jeep-vs-tank` and + either merge or sharpen the asymmetry-of-units distinction in the + second pack's briefing. + +### P3 — Map / catalog polish + +1. Author `data/maps/bridges-arena.oramap` and re-bind + `def-bridge-chokepoint` + (future) `combat-naval-amphibious-landing`. +2. Author `data/maps/chokepoint-arena.oramap` and re-bind + `def-with-ambush`, `combat-hold-chokepoint`, + `mfb-supply-line-link-between-bases`. +3. Promote the four "defensive topology" packs + (`build-defensive-tower-cluster`, `build-defensive-tower-line`, + `build-defensive-skirt-corners`, `def-in-depth-vs-single`) into a + single grouped eval-cell suite so they score one "topology-IQ" + metric. + +--- + +## Appendix — Probe methodology + +Each pack was probed by: + +```python +import openra_train # builds the engine; required for env.reset +from openra_bench.scenarios.loader import compile_level, load_pack +from openra_bench.eval_core import run_level + +pack = load_pack(pack_yaml_path) +c = compile_level(pack, "hard") +ep = run_level(c, lambda rs, Command: [Command.observe()], seed=1) +# ep.outcome ∈ {"win", "loss", "draw"}; ep.signals.game_tick = final tick. +``` + +The driver (`/tmp/stall_probe_driver.py`) invoked one subprocess per +pack so a Rust engine panic in any one pack did not abort the run. +189 packs probed in 48 seconds wall-time (Apple M-series). + +Hard tier seed=1 only. A full audit would extend to seeds 1–4 (the +documented "hard seed" range in CLAUDE.md) and to easy/medium tiers, +but seed-1 hard is the highest-pressure cell per pack — defects +visible on any tier are visible here. + +Static profile was extracted from the compiled `CompiledLevel` +(authoritative — the engine sees the merged scenario, not the raw +YAML). + +Raw probe results JSON: `/tmp/stall_probe_results.json` (regeneratable +by re-running `/tmp/stall_probe_driver.py`). diff --git a/data/maps/confined-aisle-64x40.oramap b/data/maps/confined-aisle-64x40.oramap index b2d3f1f9df945ce45006afecd72eeb3b482cfef0..a79a57f97be73691977996129747f8019fce9d6f 100644 Binary files a/data/maps/confined-aisle-64x40.oramap and b/data/maps/confined-aisle-64x40.oramap differ diff --git a/data/maps/expansion-aggro-3-base-greedy-arena.oramap b/data/maps/expansion-aggro-3-base-greedy-arena.oramap index a7e24c296b443028563d7d23474a7a0f3eb5a36e..275e3fb3e8b4d5d0ca4baa7b932735ff07880c7c 100644 Binary files a/data/maps/expansion-aggro-3-base-greedy-arena.oramap and b/data/maps/expansion-aggro-3-base-greedy-arena.oramap differ diff --git a/data/maps/gen-arena-31f6567b.oramap b/data/maps/gen-arena-31f6567b.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..53e1d4532200914f905b1a25e0a2f99492f2c01f 100644 Binary files a/data/maps/gen-arena-31f6567b.oramap and b/data/maps/gen-arena-31f6567b.oramap differ diff --git a/data/maps/gen-arena-498d46e9.oramap b/data/maps/gen-arena-498d46e9.oramap index 4bcb8ac4cdeb574b32484654ce4315da3554a704..9ee64b7d3f77932f96ee86846f25daba1d4626e9 100644 Binary files a/data/maps/gen-arena-498d46e9.oramap and b/data/maps/gen-arena-498d46e9.oramap differ diff --git a/data/maps/lh-defense-tech-second-base-arena.oramap b/data/maps/lh-defense-tech-second-base-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..53e1d4532200914f905b1a25e0a2f99492f2c01f 100644 Binary files a/data/maps/lh-defense-tech-second-base-arena.oramap and b/data/maps/lh-defense-tech-second-base-arena.oramap differ diff --git a/data/maps/mcv-deploy-third-base-arena.oramap b/data/maps/mcv-deploy-third-base-arena.oramap index 4bcb8ac4cdeb574b32484654ce4315da3554a704..4def8d14b170554723601dc1c16b88cbb099624c 100644 Binary files a/data/maps/mcv-deploy-third-base-arena.oramap and b/data/maps/mcv-deploy-third-base-arena.oramap differ diff --git a/data/maps/mfb-base-1-defend-base-2-build-arena.oramap b/data/maps/mfb-base-1-defend-base-2-build-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..11e4524f2efd06923ec024c04376714adaada385 100644 Binary files a/data/maps/mfb-base-1-defend-base-2-build-arena.oramap and b/data/maps/mfb-base-1-defend-base-2-build-arena.oramap differ diff --git a/data/maps/mfb-mirror-base-east-west-arena.oramap b/data/maps/mfb-mirror-base-east-west-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..11e4524f2efd06923ec024c04376714adaada385 100644 Binary files a/data/maps/mfb-mirror-base-east-west-arena.oramap and b/data/maps/mfb-mirror-base-east-west-arena.oramap differ diff --git a/data/maps/mfb-supply-line-corridor-arena.oramap b/data/maps/mfb-supply-line-corridor-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..d9b6676a5b876d64268c2681a0f90ffcabe93dec 100644 Binary files a/data/maps/mfb-supply-line-corridor-arena.oramap and b/data/maps/mfb-supply-line-corridor-arena.oramap differ diff --git a/data/maps/mfb-third-base-against-clock-arena.oramap b/data/maps/mfb-third-base-against-clock-arena.oramap index a0acf69723298d3a38f2919307513f6e4f6eb82d..bfff995cca951d5760ea9ba95bf75fea14681404 100644 Binary files a/data/maps/mfb-third-base-against-clock-arena.oramap and b/data/maps/mfb-third-base-against-clock-arena.oramap differ diff --git a/data/maps/mfb-two-base-simultaneous-arena.oramap b/data/maps/mfb-two-base-simultaneous-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..d9b6676a5b876d64268c2681a0f90ffcabe93dec 100644 Binary files a/data/maps/mfb-two-base-simultaneous-arena.oramap and b/data/maps/mfb-two-base-simultaneous-arena.oramap differ diff --git a/data/maps/mid-concede-arena.oramap b/data/maps/mid-concede-arena.oramap index 0e3a49b25e116f3d0c857f69871aef05352d7a99..d9b6676a5b876d64268c2681a0f90ffcabe93dec 100644 Binary files a/data/maps/mid-concede-arena.oramap and b/data/maps/mid-concede-arena.oramap differ diff --git a/data/maps/naval-arena-64x40.oramap b/data/maps/naval-arena-64x40.oramap index 158fbc36205781910f90c0bc1a3a22be2cefddbc..21d6491193146d0f50fd4e8b5895584b98bca78b 100644 Binary files a/data/maps/naval-arena-64x40.oramap and b/data/maps/naval-arena-64x40.oramap differ diff --git a/data/maps/scout-arena.oramap b/data/maps/scout-arena.oramap index 02cbf23a94846fe1a03a13e0d9ea036cd9e63563..bec6118b9f54bc27f6d2e9284b8bfa660753b754 100644 Binary files a/data/maps/scout-arena.oramap and b/data/maps/scout-arena.oramap differ diff --git a/openra_bench/agent.py b/openra_bench/agent.py index 15deafa6cbf50e3092ffe853a4510878881bbb43..60aef0025dce832eaea377d0164dd9d41858aa3d 100644 --- a/openra_bench/agent.py +++ b/openra_bench/agent.py @@ -616,6 +616,16 @@ class ModelAgent: ) self.history: list[dict] = [{"role": "system", "content": sys_content}] self.stats = {"turns": 0, "tool_calls": 0, "empty_replies": 0} + # Audit-format capture (FullPlayback). When `audit_capture` is + # True the agent stores the per-turn briefing, the provider's + # literal request/response, and exposes the system prompt so + # the audit JSONL line for the turn carries everything. + # Default off — zero overhead for normal runs. + self.audit_capture: bool = False + self.last_briefing: str = "" + self.last_request: dict | None = None + self.last_response: dict | None = None + self.system_prompt: str = sys_content # Controller contract (openra_bench/controller.py): a ModelAgent # IS a Controller — it exposes `name`, `reset`, `act` so the # eval loop, the 1v1 harness, and the human-labeling harness can @@ -768,12 +778,44 @@ class ModelAgent: def agent_fn(self, render_state: dict, Command: Any) -> list: self.stats["turns"] += 1 - self.history.append(self._user_message(render_state)) + user_msg = self._user_message(render_state) + if self.audit_capture: + # Plain-text briefing capture (image-primary turns carry a + # list `content`; the text part is the briefing). FullPlayback + # writes this as the human-readable `briefing` field so the + # audit JSONL records exactly what the model read. + c = user_msg.get("content") + if isinstance(c, str): + self.last_briefing = c + elif isinstance(c, list): + self.last_briefing = "\n".join( + p.get("text", "") for p in c + if isinstance(p, dict) and p.get("type") == "text" + ) + # Enable provider-side audit hook for this turn (drain on + # return). Lazily install the list — providers without the + # `request_log` attr (e.g. Bedrock stub) silently skip. + if hasattr(self.provider, "request_log"): + self.provider.request_log = [] + self.history.append(user_msg) self._strip_old_images(self.history) wire = self._window( self.history, getattr(self.cfg, "max_history_turns", 16) ) reply = self.provider.complete(wire, self.tools) + if self.audit_capture and hasattr(self.provider, "request_log"): + log = self.provider.request_log or [] + # One model call per turn (no internal retries here — those + # are surfaced as a single call with the eventual response); + # take the LAST entry to be safe. + if log: + self.last_request = log[-1].get("request") + self.last_response = log[-1].get("response") + else: + self.last_request = None + self.last_response = None + # Drain so the next turn starts clean. + self.provider.request_log = [] self.history.append( { "role": "assistant", diff --git a/openra_bench/eval_core.py b/openra_bench/eval_core.py index f2f9781d1d871a733b6d9ae9a46e650d8c1be3fe..8eb8ab483abedd328598352d530e66e6880fd051 100644 --- a/openra_bench/eval_core.py +++ b/openra_bench/eval_core.py @@ -230,6 +230,7 @@ def run_level( agent_fn: "AgentFn | Controller" = scripted_explore_agent, seed: int = 0, playback=None, + full_playback=None, ) -> EpisodeResult: """Run one scenario-pack level, scoring against its declarative win/fail conditions (checked every turn). Outcome maps to the @@ -272,6 +273,15 @@ def run_level( # model actually saw (same vendored _minimap_v2, accumulating). _pb_explored: set = set() _pb_terrain = None + # Audit-capture wiring: when a FullPlayback is attached, surface + # the underlying ModelAgent (if any) and flip on `audit_capture` + # so per-turn briefing / wire request+response are stashed for + # the audit JSONL. + _audit_agent = ( + introspection_source(controller) if full_playback is not None else None + ) + if _audit_agent is not None and hasattr(_audit_agent, "audit_capture"): + _audit_agent.audit_capture = True # Interrupt-driven mode (step 4): if the scenario enabled any # interrupt signals, advance with step_until_event so the agent # is re-prompted (debriefed) the moment an event fires @@ -356,6 +366,57 @@ def run_level( interrupt=interrupt, goal=turn_goal(compiled.win_condition, ctx), ) + if full_playback is not None: + # Mirror the same PNG (when the legacy playback rendered + # one). Otherwise render on-demand for the audit format. + _fp_png = locals().get("_png") if playback is not None else None + if _fp_png is None: + try: + from .minimap import terrain_png_for + if _pb_terrain is None: + _pb_terrain = terrain_png_for( + compiled.scenario.base_map + ) + from .prompt_v2 import minimap_b64 as _v2_mm + _fp_png = _v2_mm( + rs, _pb_terrain, _pb_explored, + constant_colors=compiled.level in ("easy", "medium"), + ) + if _fp_png is None: + from .agent import _render_minimap_b64 + _fp_png = _render_minimap_b64(rs, _pb_terrain) + except Exception: # noqa: BLE001 — audit never breaks a run + _fp_png = None + try: + full_playback.record_turn( + turn=turns, + tick=adapter.signals.game_tick, + obs=rs, + briefing=getattr(_audit_agent, "last_briefing", "") + if _audit_agent is not None + else "", + system_prompt=getattr(_audit_agent, "system_prompt", "") + if _audit_agent is not None + else "", + model_request=getattr(_audit_agent, "last_request", None) + if _audit_agent is not None + else None, + model_response=getattr(_audit_agent, "last_response", None) + if _audit_agent is not None + else None, + commands_issued=cmds, + engine_warnings=( + info.get("warnings", []) + if isinstance(info, dict) + else [] + ), + signals=adapter.signals, + minimap_png_b64=_fp_png, + done=bool(done), + interrupt=interrupt, + ) + except Exception: # noqa: BLE001 + pass trace.append( { "turn": turns, @@ -454,8 +515,43 @@ def run_level( }, } ) + if full_playback is not None: + try: + full_playback.finalize( + outcome=outcome, + final_obs=final_rs, + manifest_extra={ + "scenario": result.scenario, + "pack_id": compiled.pack_id, + "level": compiled.level, + "capability": compiled.meta.capability, + "seed": seed, + "outcome": outcome, + "turns": turns, + "max_turns": compiled.max_turns, + "actions_issued": issued, + "actions_warned": warned, + "agent_stats": getattr( + introspection_source(controller), "stats", None + ), + "objective_progress": result.objective_progress, + "reward_vector": result.reward_vector, + }, + ) + except Exception: # noqa: BLE001 — never break a run on I/O + pass return result finally: + # Abort the audit recorder if the loop crashed before finalize — + # leaves a `.partial` on disk for forensics; the resume scanner + # correctly sees no `.jsonl` and retries the cell. + try: + if full_playback is not None and Path( + full_playback.jsonl_path + ).exists() is False: + full_playback.abort() + except Exception: # noqa: BLE001 + pass pool.release(env) pool.shutdown() Path(tmp_path).unlink(missing_ok=True) diff --git a/openra_bench/full_playback.py b/openra_bench/full_playback.py new file mode 100644 index 0000000000000000000000000000000000000000..ea58a6437fb4243e95ef7a6040b2bed6588cf000 --- /dev/null +++ b/openra_bench/full_playback.py @@ -0,0 +1,334 @@ +"""Audit-ready per-cell data capture (Phase 4 paper-collection format). + +This is the **audit format** used by `scripts/collect_eval_data.py`. It +is additive over `openra_bench.playback.Playback`: that one stays the +inspect-by-human format (legible per-turn record, terse signals); this +one captures EVERYTHING needed to forensically replay or re-score a +run after the fact — full obs (including `_raw` and `spatial`), the +exact briefing the model saw, the system prompt, the literal HTTP +request body sent to the provider, the literal response (content + +tool_calls + finish_reason + usage), engine warnings, and a `terminal` +block on the final turn (outcome, final_obs, wall-clock, tokens). + +Layout: one JSONL per (model, pack, level, seed, fog_mode) cell at + + /____seed__.jsonl + +PNG minimaps go alongside in a sibling dir of the same stem: + + /____seed__/turn_.png + +The JSONL line for a turn refs the PNG by relative path (relative to +the JSONL file's parent). A `terminal:` field on the final line marks +the episode complete — `scripts/collect_eval_data.py --resume` uses +that marker to skip cells that already finished cleanly. + +Why a separate writer (vs extending Playback): the legacy Playback +format is what `scripts/view_playback.py` reads and what `run_eval.py` +emits today; rewriting it would invalidate every existing playback +dir and break the viewer for ~1000 historical episodes. FullPlayback +runs ALONGSIDE Playback when both are configured; either can be +disabled independently. +""" + +from __future__ import annotations + +import base64 +import json +import time +from dataclasses import asdict, is_dataclass +from pathlib import Path +from typing import Any + + +def _jsonable(o: Any) -> Any: + """Recursive serializer matching playback._jsonable but tolerant of + bytes (base64'd) and tuples (preserved as lists). Bytes are wrapped + as `{"__b64__": }` so a round-trip recovers them.""" + if isinstance(o, bytes): + return {"__b64__": base64.b64encode(o).decode("ascii")} + if is_dataclass(o) and not isinstance(o, type): + return _jsonable(asdict(o)) + if isinstance(o, dict): + return {str(k): _jsonable(v) for k, v in o.items()} + if isinstance(o, (list, tuple)): + return [_jsonable(v) for v in o] + if isinstance(o, set): + return sorted(_jsonable(v) for v in o) + if isinstance(o, (str, int, float, bool)) or o is None: + return o + return repr(o) + + +def _safe(s: str) -> str: + """Slug a model id / pack id for path use: keep alnum, `.`, `_`, `-`; + everything else (especially `/`) becomes `_`.""" + out = [] + for ch in s: + out.append(ch if (ch.isalnum() or ch in "._-") else "_") + return "".join(out) + + +def cell_stem(pack_id: str, level: str, seed: int, fog_mode: str) -> str: + return f"{_safe(pack_id)}__{_safe(level)}__seed{int(seed)}__{_safe(fog_mode)}" + + +class FullPlayback: + """Per-cell audit-ready JSONL + PNG writer. + + Construct one per (pack, level, seed, fog_mode) cell. Call + `record_turn(...)` once per model turn (mirroring how the legacy + Playback is driven from `run_level`); call `finalize(...)` on + episode end (it emits the `terminal:` field merged into the last + turn line, NOT a new line — so the file is one-line-per-turn and + the terminal marker is unambiguous). + + Concurrency: one cell == one subprocess in the collector, so no + cross-cell locking is needed. Within a process, this class is NOT + thread-safe (the eval loop is single-threaded per episode). + """ + + def __init__( + self, + root: str | Path, + pack_id: str, + level: str, + seed: int, + fog_mode: str, + ): + self.root = Path(root) + self.root.mkdir(parents=True, exist_ok=True) + self.stem = cell_stem(pack_id, level, seed, fog_mode) + self.jsonl_path = self.root / f"{self.stem}.jsonl" + self.png_dir = self.root / self.stem + self.png_dir.mkdir(parents=True, exist_ok=True) + # Use a sidecar tmp until finalize, then move atomically over the + # final path so a `--resume` scan only sees complete files. + self._tmp_path = self.root / f"{self.stem}.jsonl.partial" + self._fh = open(self._tmp_path, "w") + self.pack_id = pack_id + self.level = level + self.seed = seed + self.fog_mode = fog_mode + self._t0 = time.time() + # Buffer the last turn line so finalize() can merge `terminal:` + # into it instead of writing a fresh trailing line (one line == + # one turn; the terminal block is a field on the last line). + self._last_rec: dict | None = None + # Token accounting across the whole episode (provider-reported + # usage from each model call); the totals land in `terminal:`. + self._tokens_in = 0 + self._tokens_out = 0 + # First turn carries the system_prompt; subsequent turns repeat + # it as `null` to keep the per-turn record uniform but small. + self._sysp_written = False + + # ── per-turn ────────────────────────────────────────────────────── + + def record_turn( + self, + *, + turn: int, + tick: int | None, + obs: dict, + briefing: str, + system_prompt: str, + model_request: dict | None, + model_response: dict | None, + commands_issued: list, + engine_warnings: list[str], + signals: Any, + minimap_png_b64: str | None = None, + done: bool = False, + interrupt: str | None = None, + extra: dict | None = None, + ) -> None: + # Flush the previously-buffered line now that we know we're past + # it (a new turn started, so the prior turn was NOT terminal). + if self._last_rec is not None: + self._fh.write(json.dumps(_jsonable(self._last_rec)) + "\n") + self._fh.flush() + self._last_rec = None + + # Track per-call tokens (provider returns them in response.usage) + u = (model_response or {}).get("usage") or {} + self._tokens_in += int(u.get("prompt_tokens", 0) or 0) + self._tokens_out += int(u.get("completion_tokens", 0) or 0) + + png_rel: str | None = None + if minimap_png_b64: + try: + png_path = self.png_dir / f"turn_{int(turn):03d}.png" + png_path.write_bytes(base64.b64decode(minimap_png_b64)) + # Relative to the JSONL's parent dir so the file is + # portable (you can move the run dir without breaking + # refs). + png_rel = f"{self.stem}/turn_{int(turn):03d}.png" + except Exception: # noqa: BLE001 — never break a run on I/O + png_rel = None + + rec: dict = { + "turn": int(turn), + "tick": tick, + "interrupt": interrupt, + "obs": _jsonable(obs), + "briefing": briefing, + "system_prompt": system_prompt if not self._sysp_written else None, + "model_request": _jsonable(model_request) if model_request else None, + "model_response": _jsonable(model_response) if model_response else None, + "commands_issued": [repr(c) for c in commands_issued], + "engine_warnings": list(engine_warnings or []), + "signals": _jsonable(_signal_snapshot(signals)), + "minimap_png": png_rel, + "done": bool(done), + } + if extra: + rec["extra"] = _jsonable(extra) + self._sysp_written = True + # Buffer; finalize() merges `terminal:` into this line. + self._last_rec = rec + + def finalize( + self, + *, + outcome: str, + final_obs: dict | None, + manifest_extra: dict | None = None, + ) -> None: + """Stamp the buffered last line with the `terminal:` block, + flush, and atomically move the partial file over the final + path so `--resume` sees a fully-complete cell.""" + wall = round(time.time() - self._t0, 3) + terminal = { + "outcome": outcome, + "final_obs": _jsonable(final_obs) if final_obs is not None else None, + "wall_clock_seconds": wall, + "total_tokens_in": int(self._tokens_in), + "total_tokens_out": int(self._tokens_out), + } + if manifest_extra: + terminal["manifest"] = _jsonable(manifest_extra) + + if self._last_rec is None: + # Episode produced zero turns (engine crashed on reset, say). + # Emit a synthetic terminal-only record so `--resume` can + # still detect "this cell was attempted and completed". + self._last_rec = { + "turn": 0, + "tick": None, + "interrupt": None, + "obs": None, + "briefing": "", + "system_prompt": None, + "model_request": None, + "model_response": None, + "commands_issued": [], + "engine_warnings": [], + "signals": {}, + "minimap_png": None, + "done": True, + } + self._last_rec["terminal"] = terminal + self._fh.write(json.dumps(_jsonable(self._last_rec)) + "\n") + self._fh.flush() + try: + self._fh.close() + except Exception: # noqa: BLE001 + pass + try: + self._tmp_path.replace(self.jsonl_path) + except Exception: # noqa: BLE001 — last-ditch fallback + try: + self.jsonl_path.write_text(self._tmp_path.read_text()) + except Exception: # noqa: BLE001 + pass + + def abort(self) -> None: + """Close without finalizing. The .partial file stays on disk so + a post-hoc diagnostic can inspect what was captured before the + crash; the final .jsonl is NOT created, so `--resume` will + correctly retry this cell on the next invocation.""" + try: + if self._last_rec is not None: + self._fh.write(json.dumps(_jsonable(self._last_rec)) + "\n") + self._fh.flush() + self._fh.close() + except Exception: # noqa: BLE001 + pass + + +def _signal_snapshot(signals: Any) -> dict: + """Pull every primitive scalar / list off an EpisodeSignals (or + duck-typed shim) into a JSON-safe dict. Defensive: a missing attr + is just absent. Mirrors the existing playback shape so downstream + tools recognise the same field names, but adds a few signals + (resources, harvesters, tool_violations) that the existing playback + omits.""" + fields = ( + "game_tick", + "cash", + "resources", + "resource_capacity", + "power_provided", + "power_drained", + "harvesters", + "explored_percent", + "units_killed", + "units_lost", + "enemies_seen_ids", + "enemy_buildings_seen_ids", + "production_items", + "tool_violations", + "outcome", + ) + out: dict[str, Any] = {} + for f in fields: + if not hasattr(signals, f): + continue + v = getattr(signals, f) + if isinstance(v, (set, frozenset)): + v = sorted(_jsonable(x) for x in v) + out[f] = _jsonable(v) + # Convenience: counts + computed signals downstream uses heavily. + if "enemies_seen_ids" in out and isinstance(out["enemies_seen_ids"], list): + out["enemies_seen_count"] = len(out["enemies_seen_ids"]) + if "enemy_buildings_seen_ids" in out and isinstance( + out["enemy_buildings_seen_ids"], list + ): + out["enemy_buildings_seen_count"] = len(out["enemy_buildings_seen_ids"]) + if "cash" in out and "resources" in out: + try: + out["economy_value"] = int(out["cash"]) + int(out["resources"]) + except (TypeError, ValueError): + pass + return out + + +def is_complete_cell(jsonl_path: str | Path) -> bool: + """True iff `jsonl_path` exists, is non-empty, and the LAST line + carries a `terminal:` field. The resume scanner uses this so a + crash mid-cell (partial .jsonl with no terminal) is correctly + retried, while a cleanly finished cell is correctly skipped. + + Reads only the file tail (~64KB) — safe to call on thousands of + cells in a scan.""" + p = Path(jsonl_path) + if not p.exists() or p.stat().st_size == 0: + return False + try: + # Tail read: open, seek near end, find the last newline-bounded + # line. JSONL turn records are small (~few KB at most), so 64KB + # is plenty for the last line of even a long episode. + with open(p, "rb") as fh: + fh.seek(0, 2) + size = fh.tell() + tail_n = min(size, 65536) + fh.seek(size - tail_n) + tail = fh.read().decode("utf-8", errors="replace") + last = tail.strip().splitlines()[-1] if tail.strip() else "" + if not last: + return False + rec = json.loads(last) + except (OSError, ValueError): + return False + return isinstance(rec, dict) and "terminal" in rec diff --git a/openra_bench/playback_view.py b/openra_bench/playback_view.py index 8e524921954a34829e49f356e9e76b60c2577fa3..3bee3478e01f2cba05b04022181211b7db969f52 100644 --- a/openra_bench/playback_view.py +++ b/openra_bench/playback_view.py @@ -21,8 +21,14 @@ from pathlib import Path def load_episode(ep_dir: str | Path) -> dict: """Reassemble one ``seed`` episode folder. Tolerant of a still- - running episode (missing files become empty).""" - d = Path(ep_dir) + running episode (missing files become empty). + + Accepts EITHER a legacy seed dir OR a FullPlayback audit JSONL file; + a path ending in `.jsonl` dispatches to `load_audit_jsonl`.""" + p = Path(ep_dir) + if p.is_file() and p.suffix == ".jsonl": + return load_audit_jsonl(p) + d = p manifest = _read_json(d / "manifest.json", {}) messages = _read_json(d / "messages.json", []) turns = [] @@ -40,10 +46,115 @@ def load_episode(ep_dir: str | Path) -> dict: def find_episodes(root: str | Path) -> list[Path]: - """All ``.../seed`` episode dirs under a playback root.""" - return sorted( - p.parent for p in Path(root).glob("**/manifest.json") - ) or sorted(Path(root).glob("**/seed*")) + """All episode targets under a playback root. + + Returns a mix of two shapes the viewer transparently handles: + * Legacy `.../seed/` dirs (manifest.json + turns.jsonl + …) + * Audit-format `.../.jsonl` files written by FullPlayback — + the loader detects a `.jsonl` path and translates it on the fly. + """ + root = Path(root) + legacy = sorted( + p.parent for p in root.glob("**/manifest.json") + ) or sorted(root.glob("**/seed*")) + # Audit-format cells: `//____seedN__fog.jsonl`. + # Skip the `.partial` half-runs and any sidecar files (start with `_`). + audit = sorted( + p for p in root.glob("**/*.jsonl") + if not p.name.startswith("_") + and "__seed" in p.name + and not p.name.endswith(".partial") + ) + return legacy + audit + + +def load_audit_jsonl(jsonl_path: str | Path) -> dict: + """Translate a FullPlayback audit JSONL into the same `{dir, manifest, + turns, messages}` dict shape `load_episode` returns, so the same + viewer code (`render_streamlit` / downstream loaders) can consume + either format transparently. + + - `manifest` is reconstructed from the terminal record's + `terminal.manifest` block (+ a few top-level fields), preserving + compatibility with viewers that expect `outcome`, `model`, etc. + - `turns` mirror the legacy per-turn shape (`turn`, `tick`, + `commands`, `signals`, `goal`, `minimap_png` path). + - `messages` is synthesized from the system_prompt + each turn's + briefing/response, so the viewer's transcript pane still works. + """ + p = Path(jsonl_path) + recs: list[dict] = [] + for line in p.read_text().splitlines(): + line = line.strip() + if not line: + continue + try: + recs.append(json.loads(line)) + except json.JSONDecodeError: + continue + if not recs: + return {"dir": str(p.parent), "manifest": {}, "turns": [], "messages": []} + term_block = (recs[-1].get("terminal") or {}) if recs else {} + manifest: dict = dict(term_block.get("manifest") or {}) + # Surface the audit-only totals at the manifest level so the viewer's + # "outcome / wall / tokens" header line has them. + manifest.setdefault("outcome", term_block.get("outcome", "?")) + manifest.setdefault("wall_clock_seconds", term_block.get("wall_clock_seconds")) + manifest["total_tokens_in"] = term_block.get("total_tokens_in", 0) + manifest["total_tokens_out"] = term_block.get("total_tokens_out", 0) + turns = [] + messages: list[dict] = [] + # System prompt lives on the first turn only. + sysp = recs[0].get("system_prompt") + if sysp: + messages.append({"role": "system", "content": sysp}) + for r in recs: + # Turn record — translate to legacy keys the viewer recognises. + png_rel = r.get("minimap_png") + png_abs = str(p.parent / png_rel) if png_rel else None + sig = r.get("signals") or {} + turns.append( + { + "turn": r.get("turn"), + "tick": r.get("tick"), + "interrupt": r.get("interrupt"), + "commands": r.get("commands_issued") or [], + "ascii_minimap": (r.get("obs") or {}).get("minimap", ""), + "signals": sig, + "units": (r.get("obs") or {}).get("units_summary", []), + "enemies": (r.get("obs") or {}).get("enemy_summary", []), + "goal": {}, # not captured by FullPlayback + "minimap_png": png_abs, + } + ) + if r.get("briefing"): + messages.append({"role": "user", "content": r["briefing"]}) + resp = r.get("model_response") or {} + if resp.get("text") or resp.get("tool_calls"): + messages.append( + { + "role": "assistant", + "content": resp.get("text") or "", + "tool_calls": [ + { + "id": f"c{i}", + "type": "function", + "function": { + "name": (tc or {}).get("name", ""), + "arguments": (tc or {}).get("arguments", {}), + }, + } + for i, tc in enumerate(resp.get("tool_calls") or []) + ], + "reasoning": resp.get("reasoning", ""), + } + ) + return { + "dir": str(p.parent), + "manifest": manifest, + "turns": turns, + "messages": messages, + } def _read_json(p: Path, default): diff --git a/openra_bench/providers.py b/openra_bench/providers.py index 80723e630ff850a21487fbfc63d6ec1e3c959dde..b8881b5889007457f6bd0e9bba55f85836b22d8c 100644 --- a/openra_bench/providers.py +++ b/openra_bench/providers.py @@ -145,6 +145,11 @@ class OpenAICompatibleProvider(ChatProvider): base=cfg.retry_base_s, cap=cfg.retry_cap_s, ) + # Audit hook: when set (a list), every successful complete() + # appends a dict {"request": , "response": } so the + # FullPlayback recorder can capture the literal wire payloads. + # Drained by the caller after each turn. None disables capture. + self.request_log: list[dict] | None = None @property def cost_meter(self): @@ -222,6 +227,34 @@ class OpenAICompatibleProvider(ChatProvider): u = reply.usage or {} self._cost.add(u.get("prompt_tokens", 0), u.get("completion_tokens", 0)) self._cost.check() # raises BudgetExceeded → evaluate finalizes + if self.request_log is not None: + # Audit capture: redact the bearer header (the body is the + # interesting part) and store the literal request + raw + # response side-by-side. FullPlayback drains after each turn. + try: + self.request_log.append( + { + "request": { + "url": url, + "body": body, + }, + "response": { + "raw": reply.raw, + "text": reply.text, + "tool_calls": reply.tool_calls, + "reasoning": reply.reasoning, + "usage": dict(reply.usage or {}), + "finish_reason": ( + (reply.raw.get("choices") or [{}])[0] + .get("finish_reason") + if isinstance(reply.raw, dict) + else None + ), + }, + } + ) + except Exception: # noqa: BLE001 — audit must never break a run + pass return reply def _stream_once(self, url, headers, body) -> ChatReply: diff --git a/openra_bench/run_eval.py b/openra_bench/run_eval.py index 5d73c8b29191ff8c54af1d4dbeb66afbbf91f6ca..33731431a112d603e80d1329d9248551b24dbeb3 100644 --- a/openra_bench/run_eval.py +++ b/openra_bench/run_eval.py @@ -166,6 +166,7 @@ def evaluate( handoff_k: int = 3, handoff_bank: str | Path | None = None, repeats: int = 1, + full_playback_root: str | Path | None = None, ) -> dict: """Run packs×levels×seeds. If `held_out_seeds` is given, those are run too and tagged split='held_out'; the report adds @@ -305,10 +306,18 @@ def evaluate( for c in pack.configs ] else: - unit_iter = [ - (compile_level(pack, lv), f"{pack.meta.id}:{lv}") - for lv in levels - ] + # Apply the global fog_mode (from ProviderConfig / CLI) so a + # single-fog run can audit cells in the `image`/`structured`/ + # `-clear` channels (compiled.fog_mode defaults to vision + # without this lift, which would silently downgrade every + # cell to the canonical vision-fogged modality). + _fog = getattr(provider_cfg, "fog_mode", None) if provider_cfg else None + unit_iter = [] + for lv in levels: + cl = compile_level(pack, lv) + if _fog: + cl.fog_mode = _fog + unit_iter.append((cl, f"{pack.meta.id}:{lv}")) for compiled, cell in unit_iter: if not compiled.map_supported: skipped.append(f"{cell} (map not Rust-loadable)") @@ -333,6 +342,30 @@ def evaluate( seed, ) pb.run_id, pb.model = run_id, model + # Audit-format playback (FullPlayback): one JSONL per cell at the + # canonical `____seed__.jsonl` path the + # paper-collection script consumes. Same first-repeat gating as + # the legacy Playback. + fpb = None + if full_playback_root is not None and rep == 0: + from .full_playback import FullPlayback + + # Derive (pack_id, level, fog_mode) from the cell. For + # perception-sweep cells, the cell is `pack:level:mode`; for + # legacy/configured cells, fall back to compiled fields. + parts = cell.split(":") + _pack_id = compiled.pack_id + _level = compiled.level + _fog = getattr(compiled, "fog_mode", "vision") or "vision" + if len(parts) >= 3: + _fog = parts[-1] + fpb = FullPlayback( + Path(full_playback_root) / f"{run_id}__{_safe_model}", + pack_id=_pack_id, + level=_level, + seed=seed, + fog_mode=_fog, + ) ctrl = factory(compiled) if handoff_sweep and ":handoff-" in cell: ctrl, _hnote = _handoff_wrap( @@ -340,7 +373,7 @@ def evaluate( ) else: _hnote = "" - res = run_level(compiled, ctrl, seed=seed, playback=pb) + res = run_level(compiled, ctrl, seed=seed, playback=pb, full_playback=fpb) hstats = getattr(ctrl, "handoff_stats", None) if hstats is not None: hstats = dict(hstats) @@ -685,8 +718,22 @@ def main(argv: list[str]) -> int: "'wandb/bf16' (no fallback) — premium routing off the free pool", ) ap.add_argument("--fog-mode", default="vision", - choices=["vision", "structured"], - help="spatial channel: PNG minimap vs text fog") + choices=[ + "vision", "vision-clear", + "structured", "structured-clear", + "image", "image-clear", + ], + help="spatial channel: PNG minimap (vision), text fog " + "(structured), or image-primary (image). `-clear` " + "variants run with no fog of war.") + ap.add_argument( + "--full-playback", + default=None, + help="audit-format playback dir: one JSONL per cell at " + "/____seed__.jsonl with full obs / " + "request / response / engine warnings. Used by " + "scripts/collect_eval_data.py for paper-grade data capture.", + ) ap.add_argument("--perception-sweep", action="store_true", help="run the 2x2 perception ablation: every " "pack:level expanded into vision/structured x " @@ -757,6 +804,7 @@ def main(argv: list[str]) -> int: handoff_k=a.handoff_k, handoff_bank=a.handoff_bank, repeats=a.repeats, + full_playback_root=a.full_playback, progress=lambda d, n, rec, c: print( f"[{d}/{n}] {rec['cell']}:{rec['split']}#{rec['seed']} " f"{rec['outcome']} comp={rec['composite']} " diff --git a/openra_bench/scenarios/packs/action-multiunit-coordination.yaml b/openra_bench/scenarios/packs/action-multiunit-coordination.yaml index b89f6c4c1e15e1f111eb126638ab6e30513471f4..7f050548f81c9b5ce2e719e416ffc1e63cc625e5 100644 --- a/openra_bench/scenarios/packs/action-multiunit-coordination.yaml +++ b/openra_bench/scenarios/packs/action-multiunit-coordination.yaml @@ -55,11 +55,19 @@ base: levels: easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Two regions, 6 units. Put >=2 units in EACH region (NE ~110,6 and + # SE ~110,33) before tick 2400. One direct parallel dispatch makes + # it (~17-20 turns); touring both with one column does not. Lose if + # the deadline passes or the force is wiped. description: > - Two regions, 6 units. Put >=2 units in EACH region (NE ~110,6 and - SE ~110,33) before tick 2400. One direct parallel dispatch makes - it (~17-20 turns); touring both with one column does not. Lose if - the deadline passes or the force is wiped. + Split your six tanks into two columns and put at least two units + into EACH of the north-east and south-east enemy regions within + about 27 turns. Losing the whole force, or missing the deadline, + is a loss. overrides: {} win_condition: all_of: @@ -78,14 +86,22 @@ levels: max_turns: 30 medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three DISPERSED regions, 8 units, attrition cap. Put >=2 units in + # EACH of: NE corner (115,6), bottom-left (20,36), SE corner + # (115,34) before tick 2800, losing <=2. The regions are in three + # different directions — a single eastward column cannot reach the + # bottom-left one — so this needs genuine multi-vector dispatch, + # not one fanned-out march. Enemy infantry contest the two eastern + # lanes; the bottom-left route is clear. description: > - Three DISPERSED regions, 8 units, attrition cap. Put >=2 units in - EACH of: NE corner (115,6), bottom-left (20,36), SE corner - (115,34) before tick 2800, losing <=2. The regions are in three - different directions — a single eastward column cannot reach the - bottom-left one — so this needs genuine multi-vector dispatch, - not one fanned-out march. Enemy infantry contest the two eastern - lanes; the bottom-left route is clear. + Three regions in three different directions (north-east, south-east, + bottom-left), 8 units. Put at least two units into EACH region + within about 31 turns, losing no more than two. Enemy infantry + contest the eastern lanes. overrides: actors: # `stance: 0` (HoldFire) on every agent unit — see the base @@ -128,20 +144,28 @@ levels: # feedback loop that makes coordinate-blind grounding solvable. A # hard failure attributes to spatial grounding, not extra combat. objective_coords: relative + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three dispersed regions, 8 units, attrition cap — you are NOT + # given coordinates. Each target region is marked by a specific + # enemy building, hidden in fog until you scout it. Put >=2 units + # next to EACH of: the enemy Construction Yard in the NORTH-EAST, + # the enemy Power Plant in the SOUTH-WEST (bottom-left), and the + # enemy Ore Refinery in the SOUTH-EAST, before tick 2800, losing + # <=2. Push a group toward each direction and KEEP ADVANCING — do + # not stop at a guessed spot. The instant a unit spots a target + # building you will be interrupted and shown its exact location; + # then converge >=2 units on it. The three lie in three different + # directions; a single eastward column cannot reach the south-west + # one, so this needs genuine multi-vector dispatch. Enemy infantry + # contest the two eastern lanes; the south-west route is clear. description: > - Three dispersed regions, 8 units, attrition cap — you are NOT - given coordinates. Each target region is marked by a specific - enemy building, hidden in fog until you scout it. Put >=2 units - next to EACH of: the enemy Construction Yard in the NORTH-EAST, - the enemy Power Plant in the SOUTH-WEST (bottom-left), and the - enemy Ore Refinery in the SOUTH-EAST, before tick 2800, losing - <=2. Push a group toward each direction and KEEP ADVANCING — do - not stop at a guessed spot. The instant a unit spots a target - building you will be interrupted and shown its exact location; - then converge >=2 units on it. The three lie in three different - directions; a single eastward column cannot reach the south-west - one, so this needs genuine multi-vector dispatch. Enemy infantry - contest the two eastern lanes; the south-west route is clear. + No coordinates given. Push columns toward the north-east, + south-east, and south-west to scout for the enemy Construction + Yard, Ore Refinery, and Power Plant. Put two units beside each + within about 31 turns, losing no more than two. overrides: interrupts: # Re-prompt the moment a unit sights a marker building — gives diff --git a/openra_bench/scenarios/packs/action-sequenced-execution.yaml b/openra_bench/scenarios/packs/action-sequenced-execution.yaml index 14ff695b9c839fd65902e1ebe900033833f37f3e..7c04c212a86bcb2e26203256cf25a4527ded45e5 100644 --- a/openra_bench/scenarios/packs/action-sequenced-execution.yaml +++ b/openra_bench/scenarios/packs/action-sequenced-execution.yaml @@ -46,12 +46,20 @@ levels: # ── EASY ── one column, ONE ordered 3-waypoint route, clear lanes, # generous budget. Tests pure "go in the given order, don't idle." easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Execute ONE given route IN ORDER: waypoint W1 (mid-south, marked + # by an enemy Power Plant) → W2 (north, enemy Ore Refinery) → W3 / + # final (far SE, enemy Construction Yard). You must reach them in + # that order — rushing straight to the final does NOT count. Keep + # the column moving: arrive within the time budget or you lose. description: > - Execute ONE given route IN ORDER: waypoint W1 (mid-south, marked - by an enemy Power Plant) → W2 (north, enemy Ore Refinery) → W3 / - final (far SE, enemy Construction Yard). You must reach them in - that order — rushing straight to the final does NOT count. Keep - the column moving: arrive within the time budget or you lose. + Visit three waypoints IN ORDER: enemy Power Plant (mid-south), then + Ore Refinery (north), then Construction Yard (far south-east). + Skipping or reordering does not count. Arrive within about 27 + turns or you lose. overrides: actors: - {type: 2tnk, owner: agent, stance: 0, position: [5, 8], count: 2} @@ -80,15 +88,23 @@ levels: # execution (finish route N, then route S) overruns the budget — # only a genuine two-column parallel split clears it. Attrition cap. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # TWO given routes that must BOTH be completed, each IN ORDER. + # Route N: N1 (W, enemy Power Plant) → N2 (mid-SOUTH, enemy Ore + # Refinery) → N3 (far-E NORTH, enemy Construction Yard). Route S: + # S1 (W-south, enemy Radar Dome) → S2 (mid-NORTH, enemy Power + # Plant) → S3 (far-E SOUTH, enemy Ore Refinery). The two routes + # cross in the middle: split into two columns and run them in + # parallel — doing one fully then the other overruns the clock. + # Lose ≤1 unit; arrive within budget or lose. description: > - TWO given routes that must BOTH be completed, each IN ORDER. - Route N: N1 (W, enemy Power Plant) → N2 (mid-SOUTH, enemy Ore - Refinery) → N3 (far-E NORTH, enemy Construction Yard). Route S: - S1 (W-south, enemy Radar Dome) → S2 (mid-NORTH, enemy Power - Plant) → S3 (far-E SOUTH, enemy Ore Refinery). The two routes - cross in the middle: split into two columns and run them in - parallel — doing one fully then the other overruns the clock. - Lose ≤1 unit; arrive within budget or lose. + Run TWO three-waypoint routes IN ORDER, both required. The routes + cross at mid-map, so split into two columns and run them in + parallel. Each route is labelled by enemy buildings. Arrive within + about 34 turns and lose at most one unit. overrides: actors: # Northern staging cluster @@ -141,16 +157,24 @@ levels: # Seed picks the staging corner. Strict budget + attrition. hard: objective_coords: relative + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # TWO given routes across a large arena, each four ordered + # waypoints, described ONLY by direction — you are NOT given + # coordinates. Each waypoint is an enemy building hidden in fog: + # scout toward the stated direction and you will be interrupted the + # instant a unit sights the marker, revealing it; then move on to + # it and continue the route IN ORDER. Both routes must be completed + # (split into two columns and run them together). Keep advancing — + # idling, skipping, or doing one route then the other overruns the + # clock. Lose ≤1 unit. description: > - TWO given routes across a large arena, each four ordered - waypoints, described ONLY by direction — you are NOT given - coordinates. Each waypoint is an enemy building hidden in fog: - scout toward the stated direction and you will be interrupted the - instant a unit sights the marker, revealing it; then move on to - it and continue the route IN ORDER. Both routes must be completed - (split into two columns and run them together). Keep advancing — - idling, skipping, or doing one route then the other overruns the - clock. Lose ≤1 unit. + TWO four-waypoint routes across a large fogged arena, given only by + compass direction. Each waypoint is a hidden enemy building. Split + into two columns and run both routes IN ORDER in parallel. Arrive + within about 67 turns and lose at most one unit. overrides: # Larger arena for real fog scouting across dispersed enemy bases # (rush-hour-arena 128x40 is too cramped). Generated on demand via diff --git a/openra_bench/scenarios/packs/adv-asymmetric-weaker-must-win.yaml b/openra_bench/scenarios/packs/adv-asymmetric-weaker-must-win.yaml index d1f891bdd0931968f1780cabc31c51ca346cb41a..1345a59ebbc8d969a78fe43c8de5e8d2a909d66a 100644 --- a/openra_bench/scenarios/packs/adv-asymmetric-weaker-must-win.yaml +++ b/openra_bench/scenarios/packs/adv-asymmetric-weaker-must-win.yaml @@ -139,14 +139,21 @@ levels: # is the point of the easy tier: bare commit-and-kill, no # asymmetry pressure yet. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Two of your medium tanks (2tnk) at the west of the map (x≈8, + # y=20) must kill TWO enemy rifle infantry (e1) holding post at + # the east-centre cluster (around x=78, y=20). The infantry + # can't damage your tanks meaningfully — driving east and + # finishing them works. You MUST keep at least one tank alive + # AND retain your construction yard (`fact` at 4,20). Before + # tick 5400. Stalling loses on the clock. description: > - Two of your medium tanks (2tnk) at the west of the map (x≈8, - y=20) must kill TWO enemy rifle infantry (e1) holding post at - the east-centre cluster (around x=78, y=20). The infantry - can't damage your tanks meaningfully — driving east and - finishing them works. You MUST keep at least one tank alive - AND retain your construction yard (`fact` at 4,20). Before - tick 5400. Stalling loses on the clock. + Your two medium tanks must kill two enemy rifle infantry at the + east-centre cluster within about 60 turns. Keep at least one tank + alive and keep your construction yard standing. overrides: actors: # AGENT MEDIUMS — both on the y=20 lane; head-on works on @@ -188,22 +195,30 @@ levels: # sqrt(16²+6²) ≈ 17 > 16), so the e1s die one at a time without # ever drawing the heavy off post. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Two medium tanks (2tnk) at the west of the map must kill FOUR + # enemy rifle infantry (e1) that form a wall on the west face of + # the east-centre cluster (around x=74, y=18..22), behind which + # a HEAVY enemy tank (3tnk) holds post on the east face (90,20). + # The heavy's cannon out-DPSs your medium pair at close range — + # a head-on push along y=20 fixes you against the infantry wall + # long enough for the heavy to lunge into range and finish you. + # The only winning play is to stage OFF-AXIS (e.g. on the north + # corridor y=10), approach the infantry from a flank cell that + # is within your tank's gun range of an infantryman but OUTSIDE + # the heavy's aggro radius, pick off the infantry one at a time, + # and retreat past the heavy's leash any time it lunges. Keep + # at least one tank alive AND retain your construction yard + # (`fact` at 4,20). Before tick 5400. Stall, head-on charge, + # and brute attack_move east all LOSE. description: > - Two medium tanks (2tnk) at the west of the map must kill FOUR - enemy rifle infantry (e1) that form a wall on the west face of - the east-centre cluster (around x=74, y=18..22), behind which - a HEAVY enemy tank (3tnk) holds post on the east face (90,20). - The heavy's cannon out-DPSs your medium pair at close range — - a head-on push along y=20 fixes you against the infantry wall - long enough for the heavy to lunge into range and finish you. - The only winning play is to stage OFF-AXIS (e.g. on the north - corridor y=10), approach the infantry from a flank cell that - is within your tank's gun range of an infantryman but OUTSIDE - the heavy's aggro radius, pick off the infantry one at a time, - and retreat past the heavy's leash any time it lunges. Keep - at least one tank alive AND retain your construction yard - (`fact` at 4,20). Before tick 5400. Stall, head-on charge, - and brute attack_move east all LOSE. + Two medium tanks must kill four enemy rifle infantry walling the + east-centre cluster while a heavy enemy tank holds post just + behind them. The heavy out-trades you head-on. Keep one tank + alive, keep your construction yard, finish within about 60 turns. overrides: actors: # AGENT MEDIUMS — stage OFF-AXIS on the north corridor @@ -249,20 +264,28 @@ levels: # — a sluggish kite cadence runs the clock; the asymmetric play # has to be efficient. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Two medium tanks (2tnk) stage at one of two west-of-map + # corridors (NORTH y=12 OR SOUTH y=28, chosen by seed — + # anti-memorisation). The garrison is identical to medium — + # 4× rifle infantry (e1) walling the west face of the cluster + # (around x=74) and 1× HEAVY tank (3tnk) at the east face + # (90,20). Whichever corridor you stage on, the working flank + # attack is on YOUR side of the cluster (north flank from a + # north spawn; south flank from a south spawn). Head-on along + # y=20 still LOSES. Stage off-axis, approach the infantry on + # the flank that keeps you outside the heavy's aggro envelope, + # pick the infantry off, retreat past the leash any time the + # heavy lunges. Keep at least one tank alive AND retain your + # construction yard (`fact` at 4,20). Before tick 4500. description: > - Two medium tanks (2tnk) stage at one of two west-of-map - corridors (NORTH y=12 OR SOUTH y=28, chosen by seed — - anti-memorisation). The garrison is identical to medium — - 4× rifle infantry (e1) walling the west face of the cluster - (around x=74) and 1× HEAVY tank (3tnk) at the east face - (90,20). Whichever corridor you stage on, the working flank - attack is on YOUR side of the cluster (north flank from a - north spawn; south flank from a south spawn). Head-on along - y=20 still LOSES. Stage off-axis, approach the infantry on - the flank that keeps you outside the heavy's aggro envelope, - pick the infantry off, retreat past the leash any time the - heavy lunges. Keep at least one tank alive AND retain your - construction yard (`fact` at 4,20). Before tick 4500. + Same composition as medium but a tighter deadline (about 50 + turns), and your two tanks stage in either the north or south + corridor (seed picks). Kill four rifle infantry without losing + both tanks or your construction yard. overrides: actors: # spawn_point 0 — NORTH staging corridor (y=12). Natural diff --git a/openra_bench/scenarios/packs/adv-rps-counter-pick.yaml b/openra_bench/scenarios/packs/adv-rps-counter-pick.yaml index ecd1ec6da7b4b7c42dfbbf3062b5034bd0c2af4f..6b693190469ac72ba2935cf481b5f3d0281267a4 100644 --- a/openra_bench/scenarios/packs/adv-rps-counter-pick.yaml +++ b/openra_bench/scenarios/packs/adv-rps-counter-pick.yaml @@ -166,18 +166,25 @@ levels: # Wrong-counter (8× e3) cost-per-effect wastes on soft infantry. # Stall: kill bar (6) never met → after_ticks LOSS. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Cash $2550. The enemy is a small rifle-infantry swarm (8× e1) + # entrenched at the centre of the map. You start with a full + # base (construction yard, refinery, power, barracks, war + # factory, service depot, 2 scout jeeps) — all production + # queues are online from turn 1. Scout the enemy with a jeep, + # read the composition, then commit the whole $2550 to the + # matching counter (3× medium tanks 2tnk = $2550 walks through + # a rifle swarm; mass rockets are anti-armour ordnance and + # waste cost-per-effect on soft targets). Win when 6 enemy + # units are killed AND your construction yard still stands, + # before tick 5400. Stalling and wrong-counter both LOSE. description: > - Cash $2550. The enemy is a small rifle-infantry swarm (8× e1) - entrenched at the centre of the map. You start with a full - base (construction yard, refinery, power, barracks, war - factory, service depot, 2 scout jeeps) — all production - queues are online from turn 1. Scout the enemy with a jeep, - read the composition, then commit the whole $2550 to the - matching counter (3× medium tanks 2tnk = $2550 walks through - a rifle swarm; mass rockets are anti-armour ordnance and - waste cost-per-effect on soft targets). Win when 6 enemy - units are killed AND your construction yard still stands, - before tick 5400. Stalling and wrong-counter both LOSE. + Cash $2550, full base online. Enemy is a rifle-infantry swarm at + centre-east. Pick the matching counter and kill six enemy units + within about 60 turns, keeping your construction yard. overrides: actors: - {type: fact, owner: agent, stance: 0, position: [10, 20]} @@ -220,18 +227,26 @@ levels: # ground-truth, and a play that scouts must DO so before # committing. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Cash $2550. The enemy holds a centre garrison (~12 rifle + # infantry) on the central lane; their composition is FOGGED + # — scout with one of the starter jeeps to verify the threat + # profile BEFORE committing the budget. Your construction yard, + # refinery, power, barracks, war factory, and service depot are + # all online from turn 1. Once you've read the enemy, commit + # the WHOLE $2550 to the matching counter (3× 2tnk medium tanks + # walks through a rifle swarm; mass rockets are anti-armour and + # waste cost-per-effect; mass own rifles is a 1:1 trade with no + # positional advantage). Win when 8 enemy units are killed AND + # your construction yard still stands, before tick 5400. description: > - Cash $2550. The enemy holds a centre garrison (~12 rifle - infantry) on the central lane; their composition is FOGGED - — scout with one of the starter jeeps to verify the threat - profile BEFORE committing the budget. Your construction yard, - refinery, power, barracks, war factory, and service depot are - all online from turn 1. Once you've read the enemy, commit - the WHOLE $2550 to the matching counter (3× 2tnk medium tanks - walks through a rifle swarm; mass rockets are anti-armour and - waste cost-per-effect; mass own rifles is a 1:1 trade with no - positional advantage). Win when 8 enemy units are killed AND - your construction yard still stands, before tick 5400. + Cash $2550, full base online. A centre garrison sits in fog — + scout with a jeep first, then commit the budget to the matching + counter. Kill eight enemy units within about 60 turns, keeping + your construction yard. overrides: actors: - {type: fact, owner: agent, stance: 0, position: [10, 20]} @@ -295,20 +310,29 @@ levels: # escorts = 5, the e3 cluster has 5 rocket soldiers) while a # wrong-counter play stalls below it. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Cash $2550. Your base is fixed at the west (construction + # yard, refinery, power, barracks, war factory, service depot, + # 2 scout jeeps) — full production from turn 1. The enemy + # garrison at the centre-east of the map rotates ARCHETYPE + # across seeds: an entrenched rifle swarm (e1), a heavy tank + # column (3tnk), or an anti-armour rocket cluster (e3). Each + # archetype demands a DIFFERENT counter: tanks (2tnk) shred + # rifles; rockets (e3) shred heavy armour; mass rifles (e1) + # shred rockets (e3 are anti-armour, weak vs soft mass). + # Scout with a jeep to identify the composition BEFORE + # committing the budget — a pre-committed single build LOSES + # on at least one seed. Win when 5 enemy units are killed AND + # your construction yard still stands, before tick 5400. description: > - Cash $2550. Your base is fixed at the west (construction - yard, refinery, power, barracks, war factory, service depot, - 2 scout jeeps) — full production from turn 1. The enemy - garrison at the centre-east of the map rotates ARCHETYPE - across seeds: an entrenched rifle swarm (e1), a heavy tank - column (3tnk), or an anti-armour rocket cluster (e3). Each - archetype demands a DIFFERENT counter: tanks (2tnk) shred - rifles; rockets (e3) shred heavy armour; mass rifles (e1) - shred rockets (e3 are anti-armour, weak vs soft mass). - Scout with a jeep to identify the composition BEFORE - committing the budget — a pre-committed single build LOSES - on at least one seed. Win when 5 enemy units are killed AND - your construction yard still stands, before tick 5400. + Cash $2550, full base online. The centre-east enemy archetype + rotates per seed (rifle swarm, heavy tank column, or rocket + cluster). Scout first, then commit the budget to the matching + counter. Kill five enemy units within about 60 turns, keeping + your construction yard. overrides: actors: # ── AGENT BASE (no spawn_point — identical every seed) ── diff --git a/openra_bench/scenarios/packs/adversarial-skirmish.yaml b/openra_bench/scenarios/packs/adversarial-skirmish.yaml index bd01b6339c8e15d7bb1513318a501a137594712a..38ce722f4bf20d74da9eb38d9c3fd556ddd8af3a 100644 --- a/openra_bench/scenarios/packs/adversarial-skirmish.yaml +++ b/openra_bench/scenarios/packs/adversarial-skirmish.yaml @@ -89,11 +89,18 @@ levels: own_units_gte: 1 max_turns: 70 hard: - description: 'Rung 3 — 3 tanks vs 8, loss-capped, and the outnumbered force starts from a seed-chosen - corner (two spawn_point groups → start axis varies by seed, so a memorised kite line can''t generalise). - Enemy is mid-map beyond initial sight: engage selection under fog. - - ' + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Rung 3 — 3 tanks vs 8, loss-capped, and the outnumbered force + # starts from a seed-chosen corner (two spawn_point groups → start + # axis varies by seed, so a memorised kite line can't generalise). + # Enemy is mid-map beyond initial sight: engage selection under fog. + description: > + Outnumbered three tanks vs eight mixed enemies under fog. Start + corner varies by seed. Kill eight, losing at most two, within + about 89 turns. overrides: actors: - type: 2tnk diff --git a/openra_bench/scenarios/packs/artofwar-decoy-sacrifice.yaml b/openra_bench/scenarios/packs/artofwar-decoy-sacrifice.yaml index 411e9b04df3b25cbb5b64b96271d04573f1288f2..795ba36ab27421b92fe2f70ae179005056619653 100644 --- a/openra_bench/scenarios/packs/artofwar-decoy-sacrifice.yaml +++ b/openra_bench/scenarios/packs/artofwar-decoy-sacrifice.yaml @@ -62,13 +62,21 @@ levels: # leading the hunters off with the jeeps clears the way. Generous # clock. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A hunting picket (rocket infantry + riflemen) blocks the lane to + # the objective and chases your nearest unit. Get THREE tanks to + # the objective (far east). Driving the column straight in gets it + # shot apart before three arrive; send the fast jeeps in as bait on + # a separate vector to pull the hunters off, then run the tanks + # through. Idling or never committing loses on the clock. description: > - A hunting picket (rocket infantry + riflemen) blocks the lane to - the objective and chases your nearest unit. Get THREE tanks to - the objective (far east). Driving the column straight in gets it - shot apart before three arrive; send the fast jeeps in as bait on - a separate vector to pull the hunters off, then run the tanks - through. Idling or never committing loses on the clock. + A hunting rocket-and-rifle picket blocks the corridor to the far + east objective. Get three tanks to the objective within about 65 + turns. The picket chases the nearest unit, so an unsupported + head-on push is fatal. overrides: {} win_condition: all_of: @@ -83,10 +91,16 @@ levels: # ── MEDIUM ── +1 controlled variable: a heavier anti-armor picket # (more e3) so a head-on push is decisively fatal; tighter clock. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Heavier rocket-infantry picket — a head-on push is now decisively + # fatal. Same task: divert the hunters with the bait, land THREE + # tanks on the objective before the (tighter) deadline. description: > - Heavier rocket-infantry picket — a head-on push is now decisively - fatal. Same task: divert the hunters with the bait, land THREE - tanks on the objective before the (tighter) deadline. + Heavier rocket-infantry picket. Land three tanks on the far east + objective within about 75 turns. overrides: actors: - {type: 2tnk, owner: agent, position: [6, 20], stance: 1, count: 3} @@ -115,12 +129,19 @@ levels: # be memorised; strongest picket; strict attrition cap on top (the # main body must arrive nearly whole — only the bait may be spent). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Strongest hunting picket; the force stages from a seed-chosen + # latitude (the decoy line that works varies by seed, so it can't + # be memorised). Land THREE tanks on the objective AND lose ≤2 + # units (only the two bait jeeps may be spent — burning armour + # fails) before the tight deadline. description: > - Strongest hunting picket; the force stages from a seed-chosen - latitude (the decoy line that works varies by seed, so it can't - be memorised). Land THREE tanks on the objective AND lose ≤2 - units (only the two bait jeeps may be spent — burning armour - fails) before the tight deadline. + Strongest hunting picket. The force stages from a seed-chosen + latitude. Land three tanks on the far east objective, losing at + most two units, within about 82 turns. overrides: actors: # spawn_point 0 — northern staging diff --git a/openra_bench/scenarios/packs/artofwar-indirect-approach.yaml b/openra_bench/scenarios/packs/artofwar-indirect-approach.yaml index edd90de5369f5adda9eb5c332cff85851b1d151f..4e4befd33af5f2f3b25e12545cd434f52ef100e9 100644 --- a/openra_bench/scenarios/packs/artofwar-indirect-approach.yaml +++ b/openra_bench/scenarios/packs/artofwar-indirect-approach.yaml @@ -67,14 +67,21 @@ levels: # flank route around the wall's east end lands the force. Generous # clock. Loss-cap of 1 (a single clipped tank is forgivable here). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A leashed rocket-infantry wall blocks the direct lane east to the + # objective and shreds anything that tries to cross it. Get THREE + # tanks to the objective (far east, marked by the enemy structure). + # Driving the column straight in loses it in a turn; go the long way + # — climb to the open flank, run east PAST the end of the wall, then + # turn down to the objective. Idling never commits and loses on the + # clock. description: > - A leashed rocket-infantry wall blocks the direct lane east to the - objective and shreds anything that tries to cross it. Get THREE - tanks to the objective (far east, marked by the enemy structure). - Driving the column straight in loses it in a turn; go the long way - — climb to the open flank, run east PAST the end of the wall, then - turn down to the objective. Idling never commits and loses on the - clock. + A rocket-infantry wall blocks the direct lane east. Get three + tanks to the far east objective within about 59 turns, losing at + most one. Crossing the wall head-on is fatal. overrides: {} win_condition: all_of: @@ -92,12 +99,18 @@ levels: # east, x=26..86) so the survivable flank route is longer and the # detour costs more turns before it pays off; tighter clock; zero-loss. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # The guard wall now extends much farther east — the long way round + # is longer and costlier, and a single clipped tank now fails. Same + # task: ignore the lethal short lane entirely, take the flank route + # around the (longer) wall's end, land THREE tanks on the objective + # before the tighter deadline. description: > - The guard wall now extends much farther east — the long way round - is longer and costlier, and a single clipped tank now fails. Same - task: ignore the lethal short lane entirely, take the flank route - around the (longer) wall's end, land THREE tanks on the objective - before the tighter deadline. + Longer rocket-infantry wall. Get three tanks to the far east + objective within about 65 turns with ZERO losses. overrides: actors: - {type: 1tnk, owner: agent, position: [6, 20], stance: 1, count: 3} @@ -138,13 +151,21 @@ levels: # can't be memorised. Strongest/longest wall; strict zero-loss; every # surviving unit must arrive (all_units_in_region) and ≥3 must. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Strongest, longest wall; the force stages from a seed-chosen + # latitude (north or south of the wall), so the length and shape of + # the long way round varies by seed and can't be memorised. The + # short lane is lethal from either start. Get the WHOLE force (every + # surviving tank, ≥3) onto the objective with ZERO losses, via the + # long way around the wall's end, before the tight deadline. description: > - Strongest, longest wall; the force stages from a seed-chosen - latitude (north or south of the wall), so the length and shape of - the long way round varies by seed and can't be memorised. The - short lane is lethal from either start. Get the WHOLE force (every - surviving tank, ≥3) onto the objective with ZERO losses, via the - long way around the wall's end, before the tight deadline. + Strongest wall. Stage from a seed-chosen latitude (north or south + of the wall). Land the WHOLE surviving force (at least three + tanks) on the far east objective with ZERO losses, within about + 69 turns. overrides: actors: # spawn_point 0 — northern staging diff --git a/openra_bench/scenarios/packs/artofwar-lure-the-tiger.yaml b/openra_bench/scenarios/packs/artofwar-lure-the-tiger.yaml index ab42cdb25b3cf1951690607e0ee794f248457f1f..191e731d002a6ae7875fae58e97e13d39eed20f6 100644 --- a/openra_bench/scenarios/packs/artofwar-lure-the-tiger.yaml +++ b/openra_bench/scenarios/packs/artofwar-lure-the-tiger.yaml @@ -85,16 +85,24 @@ levels: # 2; burning a tank fails). Idling never commits and loses on the # clock. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A strong rocket-infantry guard wall blocks the one lane to the + # objective (far east, marked by the enemy structure) and snaps + # straight back to it the instant nothing threatens it. Get THREE + # tanks to the objective. Driving the column straight in runs it + # into the wall and it is destroyed; send the fast jeeps in close + # on a separate vector so a segment of the wall lunges off post + # after them, then run the tanks through the open slot before it + # snaps back. Only the two bait jeeps may be spent — burning a tank + # (≥3 lost) fails. Idling never commits and loses on the clock. description: > - A strong rocket-infantry guard wall blocks the one lane to the - objective (far east, marked by the enemy structure) and snaps - straight back to it the instant nothing threatens it. Get THREE - tanks to the objective. Driving the column straight in runs it - into the wall and it is destroyed; send the fast jeeps in close - on a separate vector so a segment of the wall lunges off post - after them, then run the tanks through the open slot before it - snaps back. Only the two bait jeeps may be spent — burning a tank - (≥3 lost) fails. Idling never commits and loses on the clock. + A leashed rocket-infantry wall blocks the lane to the far east + objective; each guard lunges at nearby foes then snaps back. Get + three tanks to the objective within about 65 turns, losing at + most two units. overrides: {} win_condition: all_of: @@ -113,12 +121,18 @@ levels: # segment off; tighter clock. Same loss cap of 2 (only the two bait # jeeps may be spent — burning armour fails). medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Denser rocket-infantry wall — a head-on push is now decisively + # fatal and the bait must pull a wider segment off the lane. Same + # task: lure a segment off post with the jeeps on a divergent + # vector, run THREE tanks through the open slot to the objective + # before the (tighter) deadline, losing only the two bait jeeps. description: > - Denser rocket-infantry wall — a head-on push is now decisively - fatal and the bait must pull a wider segment off the lane. Same - task: lure a segment off post with the jeeps on a divergent - vector, run THREE tanks through the open slot to the objective - before the (tighter) deadline, losing only the two bait jeeps. + Denser rocket-infantry wall. Get three tanks to the far east + objective within about 75 turns, losing at most two units. overrides: actors: - {type: 2tnk, owner: agent, position: [6, 20], stance: 1, count: 3} @@ -151,13 +165,20 @@ levels: # the lure line can't be memorised. Densest wall; strict loss cap of # 2 (only the two bait jeeps may be spent — burning armour fails). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Densest guard wall; the force stages from a seed-chosen latitude + # (north or south), so the bait vector and the slot the main body + # runs through vary by seed and can't be memorised. Lure a segment + # off post with the jeeps, then run THREE tanks through the open + # slot to the objective AND lose ≤2 units (only the two bait jeeps + # may be spent — burning armour fails) before the tight deadline. description: > - Densest guard wall; the force stages from a seed-chosen latitude - (north or south), so the bait vector and the slot the main body - runs through vary by seed and can't be memorised. Lure a segment - off post with the jeeps, then run THREE tanks through the open - slot to the objective AND lose ≤2 units (only the two bait jeeps - may be spent — burning armour fails) before the tight deadline. + Densest guard wall. Force stages from a seed-chosen latitude + (north or south). Land three tanks on the far east objective, + losing at most two units, within about 82 turns. overrides: actors: # spawn_point 0 — northern staging (in-bounds, y=10/14) diff --git a/openra_bench/scenarios/packs/artofwar-sequenced-citadel.yaml b/openra_bench/scenarios/packs/artofwar-sequenced-citadel.yaml index 1e262099d918365e33048627ee43aa156e00ca63..365dca23b346202cc6756a75d9887fe8f7dc0712 100644 --- a/openra_bench/scenarios/packs/artofwar-sequenced-citadel.yaml +++ b/openra_bench/scenarios/packs/artofwar-sequenced-citadel.yaml @@ -41,13 +41,21 @@ levels: # modest hold before the citadel counts. Generous clock. A beeline # straight to C never satisfies the ordered latch → timeout LOSS. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Strict ordered chain: first reach staging point A (mid-west), + # then transit point B (north-centre), then seize the citadel C + # (far east) — IN THAT ORDER. Reaching C without having passed A + # then B counts for nothing. The strike is timed: C only counts + # after the hold. A greedy beeline to C, or idling, loses on the + # clock. description: > - Strict ordered chain: first reach staging point A (mid-west), - then transit point B (north-centre), then seize the citadel C - (far east) — IN THAT ORDER. Reaching C without having passed A - then B counts for nothing. The strike is timed: C only counts - after the hold. A greedy beeline to C, or idling, loses on the - clock. + Visit staging point A (mid-west), transit point B (north-centre), + then seize the citadel at C (far east) IN ORDER. C only counts + after turn 13 and before turn 26. Light pickets contest the + route. overrides: {} win_condition: all_of: diff --git a/openra_bench/scenarios/packs/build-defensive-skirt-corners.yaml b/openra_bench/scenarios/packs/build-defensive-skirt-corners.yaml index c1b3c1642eb55d8166d5579486e7b9f3d0fb1171..97da87172a3f969a84295423ea8fc54e270937ef 100644 --- a/openra_bench/scenarios/packs/build-defensive-skirt-corners.yaml +++ b/openra_bench/scenarios/packs/build-defensive-skirt-corners.yaml @@ -161,17 +161,25 @@ levels: # (count). max_turns 60 ⇒ reachable tick 93+90·59 = 5403; # deadline 5400. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four rusher bands of rifle infantry will charge your construction + # yard (fact, at map centre (64,20)) CONCURRENTLY from the four + # diagonal corners. Build 4 pillboxes (pbox — 600cr each, budget + # exactly 2400) AND place ONE of them inside the radius-4 disc of + # EACH corner region: NE (82,9), NW (46,9), SE (82,31), SW (46,31). + # Each band spawns at one corner — a pbox planted there shreds it. + # Massing all four pillboxes on a single corner holds that corner + # but lets the other three waves reach the fact. Stall, pure-army, + # and concentrate all LOSE. The fact must survive; ≥9 enemy units + # must die before tick 5400. description: > - Four rusher bands of rifle infantry will charge your construction - yard (fact, at map centre (64,20)) CONCURRENTLY from the four - diagonal corners. Build 4 pillboxes (pbox — 600cr each, budget - exactly 2400) AND place ONE of them inside the radius-4 disc of - EACH corner region: NE (82,9), NW (46,9), SE (82,31), SW (46,31). - Each band spawns at one corner — a pbox planted there shreds it. - Massing all four pillboxes on a single corner holds that corner - but lets the other three waves reach the fact. Stall, pure-army, - and concentrate all LOSE. The fact must survive; ≥9 enemy units - must die before tick 5400. + Four rifle bands will rush your central construction yard from all + four diagonal corners. Budget $2400 — build one pillbox in each + corner approach. Keep the yard alive, kill nine enemies, within + about 60 turns. starting_cash: 2400 overrides: actors: @@ -225,17 +233,25 @@ levels: # both fails three region clauses AND lets those heavier waves walk # into the fact. max_turns 60 ⇒ reachable tick 5403; deadline 5400. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four rusher bands — 4 rifle infantry each (16 total) — will + # charge your construction yard (fact, at map centre (64,20)) + # CONCURRENTLY from the four diagonal corners. Build 4 pillboxes + # (budget exactly 2400cr = 4 pbox at 600 each) AND place ONE inside + # the radius-4 disc of EACH corner region: NE (82,9), NW (46,9), + # SE (82,31), SW (46,31). Massing all four pillboxes on a single + # corner satisfies only one region clause and lets the three + # uncovered waves walk into the fact. Stall, pure-army, and + # concentrate all LOSE. The fact must survive; ≥13 enemy units + # must die before tick 5400. description: > - Four rusher bands — 4 rifle infantry each (16 total) — will - charge your construction yard (fact, at map centre (64,20)) - CONCURRENTLY from the four diagonal corners. Build 4 pillboxes - (budget exactly 2400cr = 4 pbox at 600 each) AND place ONE inside - the radius-4 disc of EACH corner region: NE (82,9), NW (46,9), - SE (82,31), SW (46,31). Massing all four pillboxes on a single - corner satisfies only one region clause and lets the three - uncovered waves walk into the fact. Stall, pure-army, and - concentrate all LOSE. The fact must survive; ≥13 enemy units - must die before tick 5400. + Four heavier rifle bands (sixteen total) will rush your central + construction yard from all four diagonal corners. Budget $2400 — + build one pillbox in each corner approach. Keep the yard alive, + kill thirteen enemies, within about 60 turns. starting_cash: 2400 overrides: actors: @@ -292,20 +308,29 @@ levels: # must READ the fact's longitude from observation. Kill bar 13. # max_turns 60 ⇒ reachable tick 5403; deadline 5400. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # The agent construction yard (fact) flips between a WEST base + # (fact at (50,20)) and an EAST base (fact at (78,20)) by seed; the + # four corner regions of the skirt must follow. Build 4 pillboxes + # (budget 2400cr = 4 pbox at 600 each) AND place ONE inside the + # radius-4 disc of EACH corner of the CURRENT fact. For a WEST fact + # (50,20) the corners are NE (68,9) NW (32,9) SE (68,31) SW + # (32,31); for an EAST fact (78,20) they are NE (96,9) NW (60,9) + # SE (96,31) SW (60,31) — read the fact's longitude from the + # observation. Massing all four pillboxes on one corner, or + # skirting the wrong (x=64) centre, satisfies at most one region + # clause and lets the uncovered waves raze the fact. Stall, + # pure-army, and concentrate all LOSE. The fact must survive; + # ≥13 enemy units must die before tick 5400. description: > - The agent construction yard (fact) flips between a WEST base - (fact at (50,20)) and an EAST base (fact at (78,20)) by seed; the - four corner regions of the skirt must follow. Build 4 pillboxes - (budget 2400cr = 4 pbox at 600 each) AND place ONE inside the - radius-4 disc of EACH corner of the CURRENT fact. For a WEST fact - (50,20) the corners are NE (68,9) NW (32,9) SE (68,31) SW - (32,31); for an EAST fact (78,20) they are NE (96,9) NW (60,9) - SE (96,31) SW (60,31) — read the fact's longitude from the - observation. Massing all four pillboxes on one corner, or - skirting the wrong (x=64) centre, satisfies at most one region - clause and lets the uncovered waves raze the fact. Stall, - pure-army, and concentrate all LOSE. The fact must survive; - ≥13 enemy units must die before tick 5400. + Same four-corner rush, but your construction yard is at the WEST + or EAST of the map depending on seed — the four corner approaches + shift with it. Budget $2400, build one pillbox in each corner of + the current yard. Keep it alive, kill thirteen, within about 60 + turns. starting_cash: 2400 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-defensive-tower-cluster.yaml b/openra_bench/scenarios/packs/build-defensive-tower-cluster.yaml index a393c8afa4558b1a77322b0de9dfb0c25a700257..22e0ac06c0cda8a3a68f4054cc8338fa118e6b0c 100644 --- a/openra_bench/scenarios/packs/build-defensive-tower-cluster.yaml +++ b/openra_bench/scenarios/packs/build-defensive-tower-cluster.yaml @@ -133,17 +133,25 @@ levels: # fails. Stall loses (clock OR fact razed); pure-army loses (count). # max_turns 60 ⇒ reachable tick 93+90·59 = 5403; deadline 5400. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A rusher band of rifle infantry charges your construction yard + # (fact, at (10,20)). Build 4 pillboxes (pbox — 600cr each, budget + # exactly 2400) AND place at least 3 of them INSIDE the radius-4 + # disc around the fact, so their overlapping fields of fire shred + # the rush at the doorstep of the protected asset. A thin pillbox + # LINE strung along the attack lane (pboxes at x=20..35) fails the + # region clause (none land inside the radius-4 disc around the + # fact) and also lets the rifle band slip through. Stall, pure- + # army, and pbox-line all LOSE. The fact must survive; ≥3 enemy + # units must die before tick 5400. description: > - A rusher band of rifle infantry charges your construction yard - (fact, at (10,20)). Build 4 pillboxes (pbox — 600cr each, budget - exactly 2400) AND place at least 3 of them INSIDE the radius-4 - disc around the fact, so their overlapping fields of fire shred - the rush at the doorstep of the protected asset. A thin pillbox - LINE strung along the attack lane (pboxes at x=20..35) fails the - region clause (none land inside the radius-4 disc around the - fact) and also lets the rifle band slip through. Stall, pure- - army, and pbox-line all LOSE. The fact must survive; ≥3 enemy - units must die before tick 5400. + A rifle band charges your construction yard from the east. Budget + $2400 — build four pillboxes with at least three sitting within + four cells of the yard. Keep the yard alive, kill three enemies, + within about 60 turns. starting_cash: 2400 overrides: actors: @@ -194,16 +202,24 @@ levels: # 5 to require the cluster to actually fight, not just absorb. # max_turns 60 ⇒ reachable tick 5403; deadline 5400. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # The full rush band — 6 rifle infantry + 2 rocket soldiers — + # charges your construction yard (fact, at (10,20)). Build 4 + # pillboxes (budget exactly 2400cr = 4 pbox at 600 each) AND + # place at least 3 of them INSIDE the radius-4 disc around the + # fact. A pillbox LINE (1 at the fact, 3 along the lane) meets + # the count but FAILS the region bar (1 of 4 inside the disc, not + # the required 3) AND cannot mass enough firepower to blunt the + # heavier band. Stall, pure-army, and pbox-line all LOSE. The + # fact must survive; ≥5 enemy units must die before tick 5400. description: > - The full rush band — 6 rifle infantry + 2 rocket soldiers — - charges your construction yard (fact, at (10,20)). Build 4 - pillboxes (budget exactly 2400cr = 4 pbox at 600 each) AND - place at least 3 of them INSIDE the radius-4 disc around the - fact. A pillbox LINE (1 at the fact, 3 along the lane) meets - the count but FAILS the region bar (1 of 4 inside the disc, not - the required 3) AND cannot mass enough firepower to blunt the - heavier band. Stall, pure-army, and pbox-line all LOSE. The - fact must survive; ≥5 enemy units must die before tick 5400. + Heavier rush — 6 rifle infantry plus 2 rocket soldiers — charges + your construction yard. Budget $2400, build four pillboxes with + at least three within four cells of the yard. Keep the yard + alive, kill five enemies, within about 60 turns. starting_cash: 2400 overrides: actors: @@ -254,18 +270,26 @@ levels: # grows to 4 (still 2400cr, exact). Kill bar 6. # max_turns 60 ⇒ reachable tick 5403; deadline 5400. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # The agent construction yard (fact) flips between NORTH (y=14) + # and SOUTH (y=26) by seed; the cluster centre must follow. + # Build 4 pillboxes (budget 2400cr = 4 pbox at 600 each) AND + # place at least 3 of them INSIDE the radius-4 disc around the + # CURRENT fact (either (10,14) or (10,26) — read it from the + # observation). A line layout, a cluster around (10,20) (the + # old centre that no longer holds the fact), or any plan that + # doesn't follow the fact's latitude FAILS the region bar AND + # lets the rush reach the fact. Stall, pure-army, pbox-line + # all LOSE. The fact must survive; ≥6 enemy units must die + # before tick 5400. description: > - The agent construction yard (fact) flips between NORTH (y=14) - and SOUTH (y=26) by seed; the cluster centre must follow. - Build 4 pillboxes (budget 2400cr = 4 pbox at 600 each) AND - place at least 3 of them INSIDE the radius-4 disc around the - CURRENT fact (either (10,14) or (10,26) — read it from the - observation). A line layout, a cluster around (10,20) (the - old centre that no longer holds the fact), or any plan that - doesn't follow the fact's latitude FAILS the region bar AND - lets the rush reach the fact. Stall, pure-army, pbox-line - all LOSE. The fact must survive; ≥6 enemy units must die - before tick 5400. + Same rush, but your construction yard is at the north or south of + the west edge depending on seed. Budget $2400, build four + pillboxes with at least three within four cells of the current + yard. Keep it alive, kill six, within about 60 turns. starting_cash: 2400 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-defensive-tower-line.yaml b/openra_bench/scenarios/packs/build-defensive-tower-line.yaml index c293e335d1d51181b383eca1a1c47f0edd6a0e62..4f003c314d1cba9af0aeba83c72bef8dcac7dc3a 100644 --- a/openra_bench/scenarios/packs/build-defensive-tower-line.yaml +++ b/openra_bench/scenarios/packs/build-defensive-tower-line.yaml @@ -98,18 +98,26 @@ levels: # clause AND the fact razed by the rush. # max_turns 60 ⇒ reachable tick 93+90·59 = 5403; deadline 5400. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A rusher band will spawn east of the corridor and must traverse + # the narrow corridor at x=60, y=18..22 to reach your base on the + # west. Build 4 pillboxes (pbox — 600cr each, budget exactly 2400) + # AND place ONE on each of the four corridor rungs (at (60,18), + # (60,19), (60,21), (60,22)) so the rusher cannot slip past on any + # row. A cluster on the middle of the corridor satisfies the count + # but FAILS every rung and lets the rush leak through; a random + # scatter near the base fails every rung and kills nothing; a + # pure-army layout (no pbox) fails the count and lets the rush + # raze the fact. Your pillboxes must kill at least 4 of the rush; + # your fact must survive. description: > - A rusher band will spawn east of the corridor and must traverse - the narrow corridor at x=60, y=18..22 to reach your base on the - west. Build 4 pillboxes (pbox — 600cr each, budget exactly 2400) - AND place ONE on each of the four corridor rungs (at (60,18), - (60,19), (60,21), (60,22)) so the rusher cannot slip past on any - row. A cluster on the middle of the corridor satisfies the count - but FAILS every rung and lets the rush leak through; a random - scatter near the base fails every rung and kills nothing; a - pure-army layout (no pbox) fails the count and lets the rush - raze the fact. Your pillboxes must kill at least 4 of the rush; - your fact must survive. + A rush will funnel through the narrow mid-map corridor at x=60, + rows y=18 to 22. Budget $2400 — build a pillbox on each of those + four rows so nothing slips through. Kill four enemies and keep + your construction yard, within about 60 turns. starting_cash: 2400 overrides: actors: @@ -160,14 +168,21 @@ levels: # heavier wave through to fail the kill bar AND raze the fact. # max_turns 60 ⇒ reachable tick 5403; deadline 5400. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same forced rusher corridor at x=60, y=18..22. Build 4 pillboxes + # (budget 2400cr = exactly 4 pbox at 600 each) AND place ONE on + # each of the four corridor rungs (at (60,18), (60,19), (60,21), + # (60,22)). The rush wave is heavier than easy — the complete LINE + # must shred it at the corridor. Your pillboxes must kill at least + # 7 of the rush; a cluster, a scatter, and a pure-army layout all + # lose; the fact must survive. description: > - Same forced rusher corridor at x=60, y=18..22. Build 4 pillboxes - (budget 2400cr = exactly 4 pbox at 600 each) AND place ONE on - each of the four corridor rungs (at (60,18), (60,19), (60,21), - (60,22)). The rush wave is heavier than easy — the complete LINE - must shred it at the corridor. Your pillboxes must kill at least - 7 of the rush; a cluster, a scatter, and a pure-army layout all - lose; the fact must survive. + Same corridor at x=60, y=18 to 22, heavier rush wave. Budget + $2400 — build a pillbox on each of those four rows. Kill seven + enemies and keep your construction yard, within about 60 turns. starting_cash: 2400 overrides: actors: @@ -217,15 +232,23 @@ levels: # y=18..22 regardless of base latitude). max_turns 70 ⇒ reachable # tick 93+90·69 = 6303; deadline 6300. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Agent base latitude flips between NORTH (y=12) and SOUTH (y=28) + # by seed. Build 4 pillboxes (budget 2400cr = exactly 4 pbox at + # 600 each) AND place ONE on each of the four corridor rungs + # (at (60,18), (60,19), (60,21), (60,22)). The corridor at x=60 + # y=18..22 is a fixed map feature — covering the rows next to your + # base instead (y=14..18 for NORTH, y=22..26 for SOUTH) FAILS the + # rung clauses and lets the rush leak through. Your pillboxes must + # kill at least 7 of the rush; the fact must survive. description: > - Agent base latitude flips between NORTH (y=12) and SOUTH (y=28) - by seed. Build 4 pillboxes (budget 2400cr = exactly 4 pbox at - 600 each) AND place ONE on each of the four corridor rungs - (at (60,18), (60,19), (60,21), (60,22)). The corridor at x=60 - y=18..22 is a fixed map feature — covering the rows next to your - base instead (y=14..18 for NORTH, y=22..26 for SOUTH) FAILS the - rung clauses and lets the rush leak through. Your pillboxes must - kill at least 7 of the rush; the fact must survive. + Same corridor at x=60, y=18 to 22 — its location is a fixed map + feature even though your base latitude flips between north and + south by seed. Build a pillbox on each of those four rows. Kill + seven enemies and keep your yard, within about 70 turns. starting_cash: 2400 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-engineer-rebuild-after-loss.yaml b/openra_bench/scenarios/packs/build-engineer-rebuild-after-loss.yaml index 9583a79aaca9f640c1b3b1a4b6e21d0a2b742025..e0635a930b2aa7f294f6095dc8d40ae8a16988f4 100644 --- a/openra_bench/scenarios/packs/build-engineer-rebuild-after-loss.yaml +++ b/openra_bench/scenarios/packs/build-engineer-rebuild-after-loss.yaml @@ -198,19 +198,27 @@ levels: # follow-on path). Reserve 500 cr funds the powr rebuild with # $200 slack. max_turns 30 → ceiling 2703 > 2700 ✓. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You inherit a working production base — a Construction Yard + # ('fact'), an Ore Refinery ('proc'), a Power Plant ('powr'), + # a War Factory ('weap'), and one Ore Truck ('harv') with an + # ore patch — and $500 cash. The Power Plant is at LOW HEALTH + # and a single enemy heavy tank ('4tnk') is staged adjacent to + # it; the Power Plant will be DESTROYED in the opening turn. + # To recover: queue another Power Plant ('build' "powr", + # cost 300) and place it adjacent to the Construction Yard so + # production stays online. Win = the Power Plant must be + # DESTROYED and then REBUILT, AND you must still own a + # Refinery, before tick 2700. Stalling, ignoring the rebuild, + # or spending the reserve on army units all miss the bar. description: > - You inherit a working production base — a Construction Yard - ('fact'), an Ore Refinery ('proc'), a Power Plant ('powr'), - a War Factory ('weap'), and one Ore Truck ('harv') with an - ore patch — and $500 cash. The Power Plant is at LOW HEALTH - and a single enemy heavy tank ('4tnk') is staged adjacent to - it; the Power Plant will be DESTROYED in the opening turn. - To recover: queue another Power Plant ('build' "powr", - cost 300) and place it adjacent to the Construction Yard so - production stays online. Win = the Power Plant must be - DESTROYED and then REBUILT, AND you must still own a - Refinery, before tick 2700. Stalling, ignoring the rebuild, - or spending the reserve on army units all miss the bar. + You inherit a full base and $500 cash. Your low-HP Power Plant + will be destroyed on the opening turn by an adjacent enemy heavy + tank. Rebuild the Power Plant and keep your Refinery within about + 30 turns. starting_cash: 500 overrides: actors: @@ -270,20 +278,28 @@ levels: # placement cell (e.g. adjacent to the fact on the OPPOSITE side # from the strike lane). Reserve still 500 cr, deadline tightened. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same base as easy (fact, proc, low-HP powr, weap, harv, mine, + # one 3tnk defender, $500 reserve) but the enemy strike is now + # TWO heavy tanks adjacent to the Power Plant. The second tank + # guarantees the kill on the opening turn AND will engage a + # fresh Power Plant that you place too close to the strike + # lane. To recover: queue another Power Plant and place it + # ADJACENT TO THE CONSTRUCTION YARD on the far side from the + # strike lane (e.g. west of the fact). Win = the Power Plant + # must be DESTROYED and then REBUILT, AND you must still own a + # Refinery, before tick 2700. Stalling, ignoring the rebuild, + # placing the new powr in the strike lane (where the lingering + # 4tnks will destroy it again), or spending the reserve on + # army units all miss the bar. description: > - Same base as easy (fact, proc, low-HP powr, weap, harv, mine, - one 3tnk defender, $500 reserve) but the enemy strike is now - TWO heavy tanks adjacent to the Power Plant. The second tank - guarantees the kill on the opening turn AND will engage a - fresh Power Plant that you place too close to the strike - lane. To recover: queue another Power Plant and place it - ADJACENT TO THE CONSTRUCTION YARD on the far side from the - strike lane (e.g. west of the fact). Win = the Power Plant - must be DESTROYED and then REBUILT, AND you must still own a - Refinery, before tick 2700. Stalling, ignoring the rebuild, - placing the new powr in the strike lane (where the lingering - 4tnks will destroy it again), or spending the reserve on - army units all miss the bar. + Same base and $500 reserve, but TWO heavy tanks now sit beside + the low-HP Power Plant and will linger after the kill. Rebuild + the Power Plant somewhere it can survive, and keep your Refinery, + within about 30 turns. starting_cash: 500 overrides: actors: @@ -331,18 +347,25 @@ levels: # active agent assets. The active-latitude strikers do their # one-shot powr kill identically to medium. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same exogenous-loss-and-rebuild task as medium (2× 4tnk + # strike, low-HP Power Plant, $500 reserve, tick 2700 deadline) + # but your base may begin in the NORTH band (y≈14..22) OR the + # SOUTH band (y≈22..30) of the map depending on the seed. Read + # the Construction Yard's actual position from the observation + # and place the replacement Power Plant adjacent to it on the + # far side from the strike lane. A memorised "(10,18)" rebuild + # cell will mis-place out of build radius on one of the two + # spawns. Win = the Power Plant must be DESTROYED and then + # REBUILT, AND you must still own a Refinery, before tick + # 2700. The same lazy / wrong-spend plays lose as on medium. description: > - Same exogenous-loss-and-rebuild task as medium (2× 4tnk - strike, low-HP Power Plant, $500 reserve, tick 2700 deadline) - but your base may begin in the NORTH band (y≈14..22) OR the - SOUTH band (y≈22..30) of the map depending on the seed. Read - the Construction Yard's actual position from the observation - and place the replacement Power Plant adjacent to it on the - far side from the strike lane. A memorised "(10,18)" rebuild - cell will mis-place out of build radius on one of the two - spawns. Win = the Power Plant must be DESTROYED and then - REBUILT, AND you must still own a Refinery, before tick - 2700. The same lazy / wrong-spend plays lose as on medium. + Same task as medium, but your base may sit in the NORTH or SOUTH + band depending on seed. Rebuild the Power Plant adjacent to your + Construction Yard, keep your Refinery, within about 30 turns. starting_cash: 500 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-power-down-defensive.yaml b/openra_bench/scenarios/packs/build-power-down-defensive.yaml index c713775192447b18db1832d8da155f1ac46f164a..7f042e3fd3a9fdda7fbb9910e81ec04f7018df90 100644 --- a/openra_bench/scenarios/packs/build-power-down-defensive.yaml +++ b/openra_bench/scenarios/packs/build-power-down-defensive.yaml @@ -122,16 +122,24 @@ levels: # leave proc + the load-bearing structures standing. max_turns 40 → # reachable tick 93 + 90·39 = 3603 > 3600 ✓. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Your base is OVERDRAWN: 1 Power Plant (powr, +100) carries five + # drainers — proc (−30), tent (−20), barr (−20), weap (−30), + # dome (−40) = drained 140, surplus −40 (production runs at 50% + # speed in low-power). Restore power_surplus_gte:0 by powering + # DOWN any combination of NON-ESSENTIAL drainers (tent + barr + # sheds 40 exactly; weap alone sheds 30; dome alone sheds 40). + # Do NOT sell anything — proc, tent, barr, weap, dome, fact must + # remain standing. Do NOT power_down the lone powr — provided + # power must stay ≥100. Deadline tick 3600. description: > - Your base is OVERDRAWN: 1 Power Plant (powr, +100) carries five - drainers — proc (−30), tent (−20), barr (−20), weap (−30), - dome (−40) = drained 140, surplus −40 (production runs at 50% - speed in low-power). Restore power_surplus_gte:0 by powering - DOWN any combination of NON-ESSENTIAL drainers (tent + barr - sheds 40 exactly; weap alone sheds 30; dome alone sheds 40). - Do NOT sell anything — proc, tent, barr, weap, dome, fact must - remain standing. Do NOT power_down the lone powr — provided - power must stay ≥100. Deadline tick 3600. + Your base is overdrawn by 40 power. Use power_down to shed enough + non-essential load to restore non-negative surplus within about + 40 turns. Do NOT sell any building; keep at least 100 provided + power (do not power down the Power Plant). starting_cash: 1000 overrides: actors: @@ -201,17 +209,25 @@ levels: # selling, no shedding the powr). max_turns 36 → reachable tick # 3243 > 3240 ✓. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same overdrawn base, but heavier load — drainers total 180 + # (proc 30 + tent 20 + barr 20 + weap 30 + dome 40 + fix 30 + + # hpad 10 = 180). Surplus −80. You must shed ≥80 of load by + # powering DOWN non-essential drainers (e.g. weap + dome = 70 is + # NOT enough; weap + dome + tent = 90; dome + tent + barr + + # hpad = 90; dome + fix = 70 NOT enough; dome + fix + tent = + # 90). No single power_down restores surplus — combinations + # required. Do NOT sell anything — proc, tent, barr, weap, + # dome, fix, hpad, fact must remain standing. Provided must + # stay ≥100 (do not power_down the powr). Deadline tick 3240. description: > - Same overdrawn base, but heavier load — drainers total 180 - (proc 30 + tent 20 + barr 20 + weap 30 + dome 40 + fix 30 + - hpad 10 = 180). Surplus −80. You must shed ≥80 of load by - powering DOWN non-essential drainers (e.g. weap + dome = 70 is - NOT enough; weap + dome + tent = 90; dome + tent + barr + - hpad = 90; dome + fix = 70 NOT enough; dome + fix + tent = - 90). No single power_down restores surplus — combinations - required. Do NOT sell anything — proc, tent, barr, weap, - dome, fix, hpad, fact must remain standing. Provided must - stay ≥100 (do not power_down the powr). Deadline tick 3240. + Heavier overdraw (surplus -80) on a larger base. Power down + enough non-essential buildings to restore non-negative surplus + within about 36 turns. No single shed will do it. Sell nothing; + keep at least 100 provided power. starting_cash: 1000 overrides: actors: @@ -280,13 +296,21 @@ levels: # both spawn_point groups. The enemy marker has no spawn_point and # always places. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same overdrawn base as medium (surplus −90, shed ≥90 of load by + # powering DOWN), but the base latitude FLIPS between NORTH (y=12) + # and SOUTH (y=28) by seed. The building IDs you must target with + # `power_down` are at the spawn-specific coords — read them from + # the observation. Do NOT sell anything; do NOT power_down the + # powr. Deadline tick 3240. description: > - Same overdrawn base as medium (surplus −90, shed ≥90 of load by - powering DOWN), but the base latitude FLIPS between NORTH (y=12) - and SOUTH (y=28) by seed. The building IDs you must target with - `power_down` are at the spawn-specific coords — read them from - the observation. Do NOT sell anything; do NOT power_down the - powr. Deadline tick 3240. + Same overdraw as medium, but base latitude flips between north + and south by seed. Read the buildings from the observation, power + down enough to restore non-negative surplus within about 36 + turns. Sell nothing; keep at least 100 provided power. starting_cash: 1000 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-power-online-first.yaml b/openra_bench/scenarios/packs/build-power-online-first.yaml index 426d352c0e14b42a01a47a00309f0923c092121b..6dff6212c4934f9c6b60dc85fb136085521daa6d 100644 --- a/openra_bench/scenarios/packs/build-power-online-first.yaml +++ b/openra_bench/scenarios/packs/build-power-online-first.yaml @@ -128,15 +128,22 @@ levels: # the chain; the model cannot also blow the budget on a single 400- # cost mis-build (tent) and still complete the chain. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You start with a Construction Yard ('fact') at (20,20) and + # $1900. Build a Power Plant ('powr', cost 300) FIRST, then an + # Ore Refinery ('proc', cost 1400). Win = own a 'powr' AND own + # a 'proc' IN THAT ORDER (the powr must appear FIRST), before + # tick 3600. The Refinery REQUIRES a Power Plant — queuing + # 'proc' first is silently rejected by the engine. Stalling, + # building infantry/army, or attempting the Refinery before the + # Power Plant all miss the clock and LOSE. description: > - You start with a Construction Yard ('fact') at (20,20) and - $1900. Build a Power Plant ('powr', cost 300) FIRST, then an - Ore Refinery ('proc', cost 1400). Win = own a 'powr' AND own - a 'proc' IN THAT ORDER (the powr must appear FIRST), before - tick 3600. The Refinery REQUIRES a Power Plant — queuing - 'proc' first is silently rejected by the engine. Stalling, - building infantry/army, or attempting the Refinery before the - Power Plant all miss the clock and LOSE. + Start with a Construction Yard and $1900. Build a Power Plant, + then an Ore Refinery, within about 40 turns. The Refinery + requires a Power Plant first. starting_cash: 1900 overrides: actors: @@ -165,14 +172,20 @@ levels: # powr that the model rebuilds twice does drain) tightens the # margin. max_turns 32 → reachable tick 93 + 90·31 = 2883 > 2880 ✓. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same opening: 'fact' at (20,20), no power, no economy. Build + # 'powr' FIRST (cost 300), THEN 'proc' (cost 1400). Tighter + # budget ($1750 — covers the chain with $50 to spare) and + # tighter clock (tick 2880). The Refinery REQUIRES the Power + # Plant — queuing 'proc' first is silently rejected. Stalling, + # army-building, or attempting 'proc' before 'powr' all run out + # the clock and LOSE. description: > - Same opening: 'fact' at (20,20), no power, no economy. Build - 'powr' FIRST (cost 300), THEN 'proc' (cost 1400). Tighter - budget ($1750 — covers the chain with $50 to spare) and - tighter clock (tick 2880). The Refinery REQUIRES the Power - Plant — queuing 'proc' first is silently rejected. Stalling, - army-building, or attempting 'proc' before 'powr' all run out - the clock and LOSE. + Same opening with tighter cash ($1750) and a tighter clock + (about 32 turns). Build a Power Plant, then an Ore Refinery. starting_cash: 1750 overrides: actors: @@ -203,16 +216,23 @@ levels: # read from the actual fact position. max_turns 32 → reachable # tick 2883 > 2880 ✓. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same opening as medium (powr FIRST then proc, tight $1750 + # budget, tick 2880 clock) but your Construction Yard may begin + # in the NORTH band (y≈12) or the SOUTH band (y≈28) of the map + # depending on the seed. Read the fact's actual position from + # the observation and place 'powr' and 'proc' adjacent to it. + # A memorised "(20,20)" opening will mis-place the buildings + # out-of-radius on one of the two spawns. The Refinery still + # REQUIRES the Power Plant; out-of-order builds are silently + # rejected and the clock runs out as a LOSS. description: > - Same opening as medium (powr FIRST then proc, tight $1750 - budget, tick 2880 clock) but your Construction Yard may begin - in the NORTH band (y≈12) or the SOUTH band (y≈28) of the map - depending on the seed. Read the fact's actual position from - the observation and place 'powr' and 'proc' adjacent to it. - A memorised "(20,20)" opening will mis-place the buildings - out-of-radius on one of the two spawns. The Refinery still - REQUIRES the Power Plant; out-of-order builds are silently - rejected and the clock runs out as a LOSS. + Same task as medium ($1750, about 32 turns), but your Construction + Yard sits in the north or south band depending on seed. Read its + position from the observation and place buildings adjacent to it. starting_cash: 1750 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-production-throughput-multibuilding.yaml b/openra_bench/scenarios/packs/build-production-throughput-multibuilding.yaml index 6e8eee531eb11afa2bd1fd97649decc16d7911cb..b1291d3052752b17e1aba808f7945cf860da9b76 100644 --- a/openra_bench/scenarios/packs/build-production-throughput-multibuilding.yaml +++ b/openra_bench/scenarios/packs/build-production-throughput-multibuilding.yaml @@ -147,18 +147,25 @@ levels: # 6th well inside the window. max_turns 35 → reachable tick # 93 + 90·34 = 3153 > 2701 ✓ (the deadline bites as a real LOSS). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You have a full base — Construction Yard (fact), Refinery + # (proc), Power Plant (powr), Service Depot (fix) and ONE War + # Factory (weap) — and $9000. Field 6 medium tanks (2tnk, cost + # $850 each) before tick 2700. ONE war factory cannot build 6 + # tanks in time — it is a single serial production line. You have + # cash for a SECOND war factory ($2000); two factories produce + # vehicles IN PARALLEL and roughly double your output. Build the + # second weap, place it, and keep BOTH queues full. Win = own ≥6 + # medium tanks AND your fact still stands, before tick 2700. + # Stalling or spamming tanks from the single factory both miss + # the quota and LOSE on the clock. description: > - You have a full base — Construction Yard (fact), Refinery - (proc), Power Plant (powr), Service Depot (fix) and ONE War - Factory (weap) — and $9000. Field 6 medium tanks (2tnk, cost - $850 each) before tick 2700. ONE war factory cannot build 6 - tanks in time — it is a single serial production line. You have - cash for a SECOND war factory ($2000); two factories produce - vehicles IN PARALLEL and roughly double your output. Build the - second weap, place it, and keep BOTH queues full. Win = own ≥6 - medium tanks AND your fact still stands, before tick 2700. - Stalling or spamming tanks from the single factory both miss - the quota and LOSE on the clock. + Full base with one War Factory and $9000. Field six medium tanks + (2tnk) within about 30 turns, keeping your construction yard. One + factory cannot meet the deadline. starting_cash: 9000 overrides: actors: @@ -188,14 +195,20 @@ levels: # by then, comfortable WIN). Same quota, same base. max_turns 33 → # reachable tick 93 + 90·32 = 2973 > 2614 ✓. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same base as easy (fact + proc + powr + fix + ONE weap, $9000) + # but a tighter deadline: field 6 medium tanks (2tnk) before + # tick 2613. A single war factory clears only ~5 tanks by then. + # Build a SECOND war factory and feed both queues — two factories + # produce vehicles in parallel and double your throughput. Win = + # own ≥6 medium tanks AND your fact still stands, before tick + # 2613. Stall or single-factory spam both LOSE on the clock. description: > - Same base as easy (fact + proc + powr + fix + ONE weap, $9000) - but a tighter deadline: field 6 medium tanks (2tnk) before - tick 2613. A single war factory clears only ~5 tanks by then. - Build a SECOND war factory and feed both queues — two factories - produce vehicles in parallel and double your throughput. Win = - own ≥6 medium tanks AND your fact still stands, before tick - 2613. Stall or single-factory spam both LOSE on the clock. + Same base as easy with a tighter deadline. Field six medium tanks + within about 29 turns, keeping your construction yard. starting_cash: 9000 overrides: actors: @@ -228,17 +241,24 @@ levels: # the spawn latitude varies. max_turns 33 → reachable tick 2973 # > 2614 ✓. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same throughput call as medium (field 6 medium tanks before + # tick 2613 from a base with ONE pre-placed war factory; build a + # SECOND for parallel output) but your base may begin in the + # NORTH band (y≈14) or the SOUTH band (y≈26) depending on the + # seed. Read your War Factory's actual position from the + # observation and place the second weap adjacent to it. A single + # factory clears only ~5 tanks by the deadline; build the second + # and feed both queues. Win = own ≥6 medium tanks AND your fact + # still stands, before tick 2613. Stall or single-factory spam + # both LOSE on the clock. description: > - Same throughput call as medium (field 6 medium tanks before - tick 2613 from a base with ONE pre-placed war factory; build a - SECOND for parallel output) but your base may begin in the - NORTH band (y≈14) or the SOUTH band (y≈26) depending on the - seed. Read your War Factory's actual position from the - observation and place the second weap adjacent to it. A single - factory clears only ~5 tanks by the deadline; build the second - and feed both queues. Win = own ≥6 medium tanks AND your fact - still stands, before tick 2613. Stall or single-factory spam - both LOSE on the clock. + Same task as medium, but your base may sit in the north or south + band depending on seed. Field six medium tanks within about 29 + turns, keeping your construction yard. starting_cash: 9000 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-rally-point-management.yaml b/openra_bench/scenarios/packs/build-rally-point-management.yaml index 5210bd30c49dfdca14030a3b253ee6ad09d6c762..767c4001527230d217164a5700665527b586f821 100644 --- a/openra_bench/scenarios/packs/build-rally-point-management.yaml +++ b/openra_bench/scenarios/packs/build-rally-point-management.yaml @@ -135,23 +135,31 @@ levels: # (23,21)..(26,25), never enter region, never engage barr → LOSS # on the deadline. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Production logistics opening. You start with a Construction + # Yard ('fact') at (20,20), a Power Plant ('powr') at (22,22), + # and a Barracks ('tent') at (24,22) — total cash $1500. An + # enemy Barracks ('barr', the kill target) sits at the FORWARD + # staging area (62,20) ≈38 cells east of your tent. Win = at + # least 3 of your units inside a radius-5 disc around (62,20) + # AND ≥1 enemy unit/building killed AND before tick 2700. The + # tent's default rally point is right next to the tent — + # freshly-built infantry pile at (23–26, 21–25) and never + # engage. You must call set_rally_point on the tent with + # target (62,20) BEFORE (or right after) you queue the first + # Rifle Infantry ('e1', cost 100), so every subsequent unit + # walks to the forward zone and chews the enemy barr down. + # Stalling, building without setting the rally forward, or + # setting the rally to a near-base cell all miss the deadline + # and LOSE. description: > - Production logistics opening. You start with a Construction - Yard ('fact') at (20,20), a Power Plant ('powr') at (22,22), - and a Barracks ('tent') at (24,22) — total cash $1500. An - enemy Barracks ('barr', the kill target) sits at the FORWARD - staging area (62,20) ≈38 cells east of your tent. Win = at - least 3 of your units inside a radius-5 disc around (62,20) - AND ≥1 enemy unit/building killed AND before tick 2700. The - tent's default rally point is right next to the tent — - freshly-built infantry pile at (23–26, 21–25) and never - engage. You must call set_rally_point on the tent with - target (62,20) BEFORE (or right after) you queue the first - Rifle Infantry ('e1', cost 100), so every subsequent unit - walks to the forward zone and chews the enemy barr down. - Stalling, building without setting the rally forward, or - setting the rally to a near-base cell all miss the deadline - and LOSE. + You have a Construction Yard, Power Plant, and Barracks with + $1500. An enemy barracks sits at the forward zone (62,20). Get + at least three of your units into a radius-5 disc around (62,20) + and kill one enemy, within about 30 turns. starting_cash: 1500 overrides: actors: @@ -185,17 +193,24 @@ levels: # but a model that stalls for even a few turns before setting the # rally misses the window. Stall / no-rally still LOSE outright. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same opening — fact (20,20), powr (22,22), tent (24,22), + # $1500 — but the SLA tightens. Enemy 'barr' kill target at + # (62,20); win requires ≥3 of your units inside a radius-5 + # disc around (62,20) AND ≥1 kill AND before tick 1900 (tight + # — a few stall turns before the rally call will miss the + # window). Tent default rally piles units at the base; you + # must call set_rally_point on the tent with target (62,20) + # and queue infantry ('e1', cost 100) IMMEDIATELY. Stall, + # no-rally, or rally-to-a-near-base-cell all LOSE on the + # deadline. description: > - Same opening — fact (20,20), powr (22,22), tent (24,22), - $1500 — but the SLA tightens. Enemy 'barr' kill target at - (62,20); win requires ≥3 of your units inside a radius-5 - disc around (62,20) AND ≥1 kill AND before tick 1900 (tight - — a few stall turns before the rally call will miss the - window). Tent default rally piles units at the base; you - must call set_rally_point on the tent with target (62,20) - and queue infantry ('e1', cost 100) IMMEDIATELY. Stall, - no-rally, or rally-to-a-near-base-cell all LOSE on the - deadline. + Same opening with a tighter clock — about 21 turns. Get at least + three units into a radius-5 disc around the forward zone (62,20) + and kill one enemy. starting_cash: 1500 overrides: actors: @@ -228,20 +243,28 @@ levels: # the centreline (62,20)). Same tight clock as medium. max_turns # 22 → reachable tick 1983 > 1900 ✓. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same SLA-tight opening as medium (forward zone (62,20), + # enemy 'barr' there, ≥3 units in r=5 disc + ≥1 kill before + # tick 1900) but your Construction Yard, Power Plant, and + # Barracks may begin in the NORTH band (y≈16) or the SOUTH + # band (y≈28) of the map depending on the seed. Read the + # tent's actual position from the observation and call + # set_rally_point on the tent with target (62,20) — the + # centreline forward zone is fixed regardless of which spawn + # you got. A memorised "set rally from y=22" opening will + # still target (62,20), but your tent id and starting cells + # will differ per seed. The Barracks default rally piles + # units at the base; without an explicit forward rally call + # the units never reach (62,20) and the SLA is missed. description: > - Same SLA-tight opening as medium (forward zone (62,20), - enemy 'barr' there, ≥3 units in r=5 disc + ≥1 kill before - tick 1900) but your Construction Yard, Power Plant, and - Barracks may begin in the NORTH band (y≈16) or the SOUTH - band (y≈28) of the map depending on the seed. Read the - tent's actual position from the observation and call - set_rally_point on the tent with target (62,20) — the - centreline forward zone is fixed regardless of which spawn - you got. A memorised "set rally from y=22" opening will - still target (62,20), but your tent id and starting cells - will differ per seed. The Barracks default rally piles - units at the base; without an explicit forward rally call - the units never reach (62,20) and the SLA is missed. + Same task as medium, but your base sits in the north or south + band depending on seed. The forward zone (62,20) is the same + either way. Get three units there and kill one enemy within + about 21 turns. starting_cash: 1500 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-repair-priority-under-fire.yaml b/openra_bench/scenarios/packs/build-repair-priority-under-fire.yaml index 46f485600b0dceedc78b774c6bcd8511dcbcaa75..0eb381443433a5ac940895c5fe8b164c4906bb29 100644 --- a/openra_bench/scenarios/packs/build-repair-priority-under-fire.yaml +++ b/openra_bench/scenarios/packs/build-repair-priority-under-fire.yaml @@ -181,20 +181,28 @@ levels: # 2400]; max_turns 28 → reachable tick 93 + 90·27 = 2523, so the # after_ticks 2401 fail bites. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three of your structures are under attrition: the refinery + # (proc), the war factory (weap), and the pillbox (pbox). The + # pillbox is already damaged to about 30% HP and looks the most + # damaged — but it is low value, heavily armoured, and the + # grenadiers barely scratch it; it survives on its own. The + # refinery is on a lethal trajectory: it dies within a few turns + # unless you toggle REPAIR on it now. Repairing the refinery also + # keeps its attackers pinned, so the war factory stays safe. Win + # when the refinery and the war factory are both still standing + # after tick 1200 and before tick 2400. If the refinery falls, or + # the clock runs out with the win unmet, you lose. Stalling and + # repairing the most-damaged-looking building (the pillbox) both + # lose the refinery. description: > - Three of your structures are under attrition: the refinery - (proc), the war factory (weap), and the pillbox (pbox). The - pillbox is already damaged to about 30% HP and looks the most - damaged — but it is low value, heavily armoured, and the - grenadiers barely scratch it; it survives on its own. The - refinery is on a lethal trajectory: it dies within a few turns - unless you toggle REPAIR on it now. Repairing the refinery also - keeps its attackers pinned, so the war factory stays safe. Win - when the refinery and the war factory are both still standing - after tick 1200 and before tick 2400. If the refinery falls, or - the clock runs out with the win unmet, you lose. Stalling and - repairing the most-damaged-looking building (the pillbox) both - lose the refinery. + Three of your buildings — refinery, war factory, and pillbox — + are taking fire. The pillbox starts at 30% HP. Use repair to + keep the refinery and war factory both alive from turn 14 to + about turn 27. overrides: actors: # proc — HIGH value, LETHAL trajectory. 3 grenadiers kill the @@ -241,18 +249,26 @@ levels: # max_turns 31 → reachable tick 93 + 90·30 = 2793, so after_ticks # 2701 bites. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three structures are under attrition: the refinery (proc), the + # war factory (weap), and the pillbox (pbox). The pillbox is + # already damaged to about 30% HP and looks the most damaged — but + # it is low value, heavily armoured, and survives on its own. The + # refinery AND the war factory are both on lethal trajectories: + # each dies within a few turns unless you toggle REPAIR on it. Win + # when the refinery and the war factory are both still standing + # after tick 1700 and before tick 2700. If either falls, or the + # clock runs out with the win unmet, you lose. Stalling, repairing + # only the pillbox, and repairing only one of the two critical + # buildings all lose. description: > - Three structures are under attrition: the refinery (proc), the - war factory (weap), and the pillbox (pbox). The pillbox is - already damaged to about 30% HP and looks the most damaged — but - it is low value, heavily armoured, and survives on its own. The - refinery AND the war factory are both on lethal trajectories: - each dies within a few turns unless you toggle REPAIR on it. Win - when the refinery and the war factory are both still standing - after tick 1700 and before tick 2700. If either falls, or the - clock runs out with the win unmet, you lose. Stalling, repairing - only the pillbox, and repairing only one of the two critical - buildings all lose. + Same three buildings under fire, but now both the refinery and + war factory are on lethal trajectories. The pillbox is the + pre-damaged decoy. Keep the refinery and war factory alive from + turn 19 to about turn 30. overrides: actors: - {type: proc, owner: agent, position: [20, 8]} @@ -292,19 +308,26 @@ levels: # agent unit (the base is otherwise building-only). Same composition # and clock as medium (proc + weap both lethal; pbox decoy). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three structures — the refinery (proc), the war factory (weap), + # and the pillbox (pbox) — are under attrition from three grenadier + # bands. Your base stages from a seed-chosen longitude, so a single + # memorised opening cannot generalise. The pillbox is pre-damaged + # to ~30% HP and looks the most damaged but is low value, heavily + # armoured, and survives on its own; the refinery and the war + # factory are both on lethal trajectories and each needs REPAIR. + # Win when the refinery and the war factory are both still standing + # after tick 1700 and before tick 2700. If either falls, or the + # clock runs out with the win unmet, you lose. Stalling, repairing + # only the pillbox, and repairing only one critical building all + # lose. description: > - Three structures — the refinery (proc), the war factory (weap), - and the pillbox (pbox) — are under attrition from three grenadier - bands. Your base stages from a seed-chosen longitude, so a single - memorised opening cannot generalise. The pillbox is pre-damaged - to ~30% HP and looks the most damaged but is low value, heavily - armoured, and survives on its own; the refinery and the war - factory are both on lethal trajectories and each needs REPAIR. - Win when the refinery and the war factory are both still standing - after tick 1700 and before tick 2700. If either falls, or the - clock runs out with the win unmet, you lose. Stalling, repairing - only the pillbox, and repairing only one critical building all - lose. + Same task as medium, but your base sits at a seed-chosen + longitude. Keep the refinery and war factory alive from turn 19 + to about turn 30. overrides: actors: # WEST base group (spawn_point 0). Full base + bands duplicated diff --git a/openra_bench/scenarios/packs/build-sell-and-rebuild-elsewhere.yaml b/openra_bench/scenarios/packs/build-sell-and-rebuild-elsewhere.yaml index 33e49da2c5262eee868e48a4b19e3ec99513ee5d..1669bd14aa8b4444a56bcf918cf56affc9e3ebe0 100644 --- a/openra_bench/scenarios/packs/build-sell-and-rebuild-elsewhere.yaml +++ b/openra_bench/scenarios/packs/build-sell-and-rebuild-elsewhere.yaml @@ -163,20 +163,28 @@ levels: # paired with after_ticks 4501 in fail ⇒ a non-finisher is a real # reachable timeout LOSS (not a draw). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You own a Construction Yard (fact) and a Power Plant (powr) + # at the safe far north-west corner and a forward Refinery + # (proc) at the centre lane. A small hunt band (2 rifle + # infantry) is incoming from the east on the centre lane and + # will raze the refinery within ~25-30 turns. Your starting + # cash is 800 — not enough to build a new refinery (cost + # 1400). SELL the exposed refinery (refunds 700) and use the + # recouped cash + starting cash to BUILD a new refinery at + # the safe target region around (16, 8) — north of the rush + # lane. Win by having a refinery at the safe region AND the + # Construction Yard still alive AND before tick 4500. Stall, + # build-without-selling (cash gated), or placing the new + # refinery anywhere outside the safe region all lose. description: > - You own a Construction Yard (fact) and a Power Plant (powr) - at the safe far north-west corner and a forward Refinery - (proc) at the centre lane. A small hunt band (2 rifle - infantry) is incoming from the east on the centre lane and - will raze the refinery within ~25-30 turns. Your starting - cash is 800 — not enough to build a new refinery (cost - 1400). SELL the exposed refinery (refunds 700) and use the - recouped cash + starting cash to BUILD a new refinery at - the safe target region around (16, 8) — north of the rush - lane. Win by having a refinery at the safe region AND the - Construction Yard still alive AND before tick 4500. Stall, - build-without-selling (cash gated), or placing the new - refinery anywhere outside the safe region all lose. + Your forward refinery is on the centre lane in the path of an + incoming rifle band; $800 alone cannot fund a new refinery. End + up with a refinery near (16,8) in the safe north-west corner and + keep your construction yard, within about 50 turns. starting_cash: 800 overrides: actors: @@ -231,19 +239,26 @@ levels: # of ~25-30. Same tick budget so the win window is tighter against # the same after_ticks 5401 fail. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You own a Construction Yard and a Power Plant at the safe + # far north-west corner and a forward Refinery at the centre + # lane. A heavier hunt band (3 rifle infantry) is incoming + # and will raze the refinery faster (~20 turns). Your + # starting cash is 700 — exactly the sell refund of a + # refinery, half the build cost. You MUST sell the exposed + # refinery to free the second half of the cash, then build a + # new refinery at the safe target region around (16, 8). Win + # by having a refinery at the safe region AND the Construction + # Yard still alive AND before tick 4500. Stalling, building + # without selling (cash blocks the build), or placing the new + # refinery in the central lane all lose. description: > - You own a Construction Yard and a Power Plant at the safe - far north-west corner and a forward Refinery at the centre - lane. A heavier hunt band (3 rifle infantry) is incoming - and will raze the refinery faster (~20 turns). Your - starting cash is 700 — exactly the sell refund of a - refinery, half the build cost. You MUST sell the exposed - refinery to free the second half of the cash, then build a - new refinery at the safe target region around (16, 8). Win - by having a refinery at the safe region AND the Construction - Yard still alive AND before tick 4500. Stalling, building - without selling (cash blocks the build), or placing the new - refinery in the central lane all lose. + Heavier rifle band on the centre lane and only $700 cash. End up + with a refinery near (16,8) in the safe north-west corner and + keep your construction yard, within about 50 turns. starting_cash: 700 overrides: actors: @@ -276,20 +291,28 @@ levels: # (16, 8)" cell loses on the SOUTH spawn (the safe region there # is (16, 36) not (16, 8)). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Your base stages from a seed-chosen latitude (NORTH y=4 or + # SOUTH y=36) — a single memorised target cell cannot + # generalise. You own a Construction Yard and a Power Plant + # at the safe corner of your latitude and a forward Refinery + # at the centre lane. Hunt bands are incoming on the centre + # lane and will raze the refinery within ~20 turns. Your + # starting cash is 700 — exactly half the refinery build + # cost. You MUST sell the exposed refinery and use the + # recouped cash to build a new refinery at the safe target + # region of your OWN latitude (around (16, 8) for the NORTH + # spawn, (16, 36) for the SOUTH spawn). Win by having a + # refinery at the safe region of your latitude AND the + # Construction Yard still alive AND before tick 4500. description: > - Your base stages from a seed-chosen latitude (NORTH y=4 or - SOUTH y=36) — a single memorised target cell cannot - generalise. You own a Construction Yard and a Power Plant - at the safe corner of your latitude and a forward Refinery - at the centre lane. Hunt bands are incoming on the centre - lane and will raze the refinery within ~20 turns. Your - starting cash is 700 — exactly half the refinery build - cost. You MUST sell the exposed refinery and use the - recouped cash to build a new refinery at the safe target - region of your OWN latitude (around (16, 8) for the NORTH - spawn, (16, 36) for the SOUTH spawn). Win by having a - refinery at the safe region of your latitude AND the - Construction Yard still alive AND before tick 4500. + Same task as medium, but your base sits in the north or south + corner depending on seed. End up with a refinery on your own + latitude's safe shoulder and keep your construction yard, within + about 50 turns. starting_cash: 700 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-sequence-tech-cheapest.yaml b/openra_bench/scenarios/packs/build-sequence-tech-cheapest.yaml index 244a9e18fe33663c70dd5e344e2b48be4a4e418d..2d8d00410b9e813b8848f9d97e1242eb2a2affa2 100644 --- a/openra_bench/scenarios/packs/build-sequence-tech-cheapest.yaml +++ b/openra_bench/scenarios/packs/build-sequence-tech-cheapest.yaml @@ -125,17 +125,25 @@ levels: # the `then:` chain never completes ⇒ after_ticks LOSS. Stall never # builds anything ⇒ LOSS on the same clause. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) by spending the LEAST cash, on the + # ONLY affordable prerequisite chain: powr → proc → weap. Your + # starting cash ($3750) is a fixed budget — there is no ore and no + # income, so it is all the money you will ever have. The minimal + # chain costs exactly $3700 (powr $300 + proc $1400 + weap $2000). + # Any extra structure bought before weap (a barracks/tent $500, a + # pillbox $600, an infantry unit) exhausts the budget and weap can + # never be funded — you LOSE on the clock. The `then:` chain + # enforces the exact order; placing weap before proc cannot satisfy + # it (and the engine refuses too: weap's prerequisite is proc). description: > - Build a war factory (weap) by spending the LEAST cash, on the - ONLY affordable prerequisite chain: powr → proc → weap. Your - starting cash ($3750) is a fixed budget — there is no ore and no - income, so it is all the money you will ever have. The minimal - chain costs exactly $3700 (powr $300 + proc $1400 + weap $2000). - Any extra structure bought before weap (a barracks/tent $500, a - pillbox $600, an infantry unit) exhausts the budget and weap can - never be funded — you LOSE on the clock. The `then:` chain - enforces the exact order; placing weap before proc cannot satisfy - it (and the engine refuses too: weap's prerequisite is proc). + Cash $3750 is a fixed budget — no ore, no income. End up owning a + Power Plant, then an Ore Refinery, then a War Factory, in that + order, within about 36 turns. Any other building drains the + budget below what the war factory needs. starting_cash: 3750 overrides: actors: @@ -169,15 +177,21 @@ levels: # chain still fits; ANY wasteful spend overruns even harder. Same # generous clock T = 3200 — money, not time, remains the teeth. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) on the cost-minimal prerequisite + # chain: powr → proc → weap. Tighter budget — your starting cash + # ($3720) barely covers the minimal path ($3700: powr $300 + + # proc $1400 + weap $2000). There is no ore and no income. Any + # extra structure (tent / pbox / an infantry unit) bought before + # weap exhausts the budget and weap can never be funded. The + # `then:` chain enforces the exact order; weap before proc cannot + # satisfy it. description: > - Build a war factory (weap) on the cost-minimal prerequisite - chain: powr → proc → weap. Tighter budget — your starting cash - ($3720) barely covers the minimal path ($3700: powr $300 + - proc $1400 + weap $2000). There is no ore and no income. Any - extra structure (tent / pbox / an infantry unit) bought before - weap exhausts the budget and weap can never be funded. The - `then:` chain enforces the exact order; weap before proc cannot - satisfy it. + Same task with a tighter $3720 budget. Build Power Plant, then + Refinery, then War Factory, in order, within about 36 turns. starting_cash: 3720 overrides: actors: @@ -208,15 +222,22 @@ levels: # do NOT honour spawn_point (CLAUDE.md), so the lone enemy `fact` # landmark always places. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) on the cost-minimal prerequisite + # chain: powr → proc → weap, from a seed-chosen base (NORTH or + # SOUTH). Tight budget — starting cash ($3720) barely covers the + # minimal path ($3700). There is no ore and no income. Any extra + # structure (tent / pbox / an infantry unit) bought before weap + # exhausts the budget and weap can never be funded. Placement + # that memorises one spawn's geometry cannot generalise; compute + # placement relative to your actual fact each run. description: > - Build a war factory (weap) on the cost-minimal prerequisite - chain: powr → proc → weap, from a seed-chosen base (NORTH or - SOUTH). Tight budget — starting cash ($3720) barely covers the - minimal path ($3700). There is no ore and no income. Any extra - structure (tent / pbox / an infantry unit) bought before weap - exhausts the budget and weap can never be funded. Placement - that memorises one spawn's geometry cannot generalise; compute - placement relative to your actual fact each run. + Same task as medium, but your base sits in the north or south + band depending on seed. Build the powr→proc→weap chain in order + within about 36 turns on $3720. starting_cash: 3720 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-sequence-tech-fastest.yaml b/openra_bench/scenarios/packs/build-sequence-tech-fastest.yaml index 805f1f2d31cfe9d0cb45ead7bbcd0c136d03934e..cf95d8db0b611f3461fe5c7305a9b2c99df9d5d7 100644 --- a/openra_bench/scenarios/packs/build-sequence-tech-fastest.yaml +++ b/openra_bench/scenarios/packs/build-sequence-tech-fastest.yaml @@ -98,15 +98,22 @@ levels: # turns) finishes at ~tick 3063, beyond T ⇒ LOSS. Stall finishes # never ⇒ LOSS on the after_ticks fail clause. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) as fast as possible by following the + # ONLY cost-optimal prerequisite chain: powr → proc → weap. Any + # detour (a barracks/tent, a redundant power plant, an early + # infantry training queue) wastes the budget and you LOSE on the + # clock. The `then:` chain enforces the exact order — placing + # weap before proc cannot satisfy it (and the engine refuses too: + # weap's prerequisite is proc). Optimal play finishes by tick + # ~2613; the deadline is 3000. description: > - Build a war factory (weap) as fast as possible by following the - ONLY cost-optimal prerequisite chain: powr → proc → weap. Any - detour (a barracks/tent, a redundant power plant, an early - infantry training queue) wastes the budget and you LOSE on the - clock. The `then:` chain enforces the exact order — placing - weap before proc cannot satisfy it (and the engine refuses too: - weap's prerequisite is proc). Optimal play finishes by tick - ~2613; the deadline is 3000. + $5000 cash plus ore. End up owning a Power Plant, then an Ore + Refinery, then a War Factory, in that order, within about 34 + turns. Detours through other buildings overrun the clock. starting_cash: 5000 overrides: actors: @@ -146,12 +153,18 @@ levels: # No additional pieces — the SAME cost-optimal chain, executed # with less slack. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) on the cost-optimal prerequisite + # chain: powr → proc → weap. Tighter deadline (2800 ticks) — any + # detour (tent / second powr / infantry queue) makes you miss. + # The `then:` chain enforces the exact order; weap before proc + # cannot satisfy it. Optimal play finishes by tick ~2613. description: > - Build a war factory (weap) on the cost-optimal prerequisite - chain: powr → proc → weap. Tighter deadline (2800 ticks) — any - detour (tent / second powr / infantry queue) makes you miss. - The `then:` chain enforces the exact order; weap before proc - cannot satisfy it. Optimal play finishes by tick ~2613. + Same task with a tighter clock (about 31 turns). Build Power + Plant, then Refinery, then War Factory, in order. starting_cash: 5000 overrides: actors: @@ -184,13 +197,20 @@ levels: # actors do NOT honour spawn_point (CLAUDE.md), so the lone # enemy `fact` always places. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a war factory (weap) on the cost-optimal prerequisite + # chain: powr → proc → weap, from a seed-chosen base (NORTH or + # SOUTH). Tight 2800-tick deadline — detours (tent / extra + # powr / infantry queue) lose on the clock. Placement that + # memorises one spawn's geometry cannot generalise; compute + # placement relative to your actual fact each run. description: > - Build a war factory (weap) on the cost-optimal prerequisite - chain: powr → proc → weap, from a seed-chosen base (NORTH or - SOUTH). Tight 2800-tick deadline — detours (tent / extra - powr / infantry queue) lose on the clock. Placement that - memorises one spawn's geometry cannot generalise; compute - placement relative to your actual fact each run. + Same task as medium, but your base sits in the north or south + band depending on seed. Build powr→proc→weap in order within + about 31 turns. starting_cash: 5000 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-sequence-tech-most-resilient.yaml b/openra_bench/scenarios/packs/build-sequence-tech-most-resilient.yaml index e8e6b18dabcc2cdcf2071ab52b963cd9aee856fd..c34e87a93dc9a6ff54c26a4146c85e7935b4496b 100644 --- a/openra_bench/scenarios/packs/build-sequence-tech-most-resilient.yaml +++ b/openra_bench/scenarios/packs/build-sequence-tech-most-resilient.yaml @@ -172,24 +172,32 @@ levels: # tanks. Generous clock (within_ticks 5400, max_turns 60 → ceiling # 5403 ✓). The strike fires at tick 1500. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You inherit a partial base — a Construction Yard ('fact'), an + # Ore Refinery ('proc'), a Service Depot ('fix'), an Ore Truck + # ('harv') with an ore patch, and ONE Power Plant ('powr'). That + # power plant sits FORWARD at the east edge of your base and is + # EXPOSED: an enemy strike will RAZE it at tick 1500. It is your + # only power. If it is your only power when the strike lands, + # your grid goes negative, your war factory drops to half + # production speed, and your tank army cannot finish in time. To + # stay resilient: build a SECOND Power Plant ('build' "powr", + # cost 300) and place it next to your Construction Yard in the + # safe west base BEFORE tick 1500, build a War Factory ('build' + # "weap", cost 2000), then produce three medium tanks ('build' + # "2tnk", cost 850 each). WIN = you brought power then a war + # factory online, you still own a Power Plant, you have 3 medium + # tanks, and you still own your Construction Yard, before tick + # 5400. Stalling, or relying on the single exposed power plant + # with no redundant backup, misses the bar. description: > - You inherit a partial base — a Construction Yard ('fact'), an - Ore Refinery ('proc'), a Service Depot ('fix'), an Ore Truck - ('harv') with an ore patch, and ONE Power Plant ('powr'). That - power plant sits FORWARD at the east edge of your base and is - EXPOSED: an enemy strike will RAZE it at tick 1500. It is your - only power. If it is your only power when the strike lands, - your grid goes negative, your war factory drops to half - production speed, and your tank army cannot finish in time. To - stay resilient: build a SECOND Power Plant ('build' "powr", - cost 300) and place it next to your Construction Yard in the - safe west base BEFORE tick 1500, build a War Factory ('build' - "weap", cost 2000), then produce three medium tanks ('build' - "2tnk", cost 850 each). WIN = you brought power then a war - factory online, you still own a Power Plant, you have 3 medium - tanks, and you still own your Construction Yard, before tick - 5400. Stalling, or relying on the single exposed power plant - with no redundant backup, misses the bar. + You inherit a base with $6000 and one forward Power Plant that an + enemy strike razes at turn 17. End up owning a Power Plant, a + War Factory, and three medium tanks (2tnk) within about 60 turns, + keeping your Construction Yard. starting_cash: 6000 overrides: actors: @@ -252,23 +260,30 @@ levels: # before committing the redundant powr now risks the deadline. The # single-point-of-failure failure modes lose exactly as on easy. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same inherited base as easy — a Construction Yard, an Ore + # Refinery, a Service Depot, an Ore Truck with an ore patch, and + # ONE EXPOSED Power Plant forward at the east edge that an enemy + # strike will RAZE at tick 1500. Build a SECOND Power Plant + # ('build' "powr", 300) in the safe west base next to your + # Construction Yard BEFORE tick 1500, build a War Factory + # ('build' "weap", 2000), then produce three medium tanks + # ('build' "2tnk", 850 each). + # The deadline is tighter — tick 4500 — so commit the redundant + # power plant early; do not wait for the exposed one to fall. If + # the strike leaves you with no power, the war factory halves its + # output and the army misses the clock. WIN = you brought power + # then a war factory online, you still own a Power Plant, you + # have 3 medium tanks, and you still own your Construction Yard, + # before tick 4500. Stalling, or relying on the single exposed + # power plant, misses the bar. description: > - Same inherited base as easy — a Construction Yard, an Ore - Refinery, a Service Depot, an Ore Truck with an ore patch, and - ONE EXPOSED Power Plant forward at the east edge that an enemy - strike will RAZE at tick 1500. Build a SECOND Power Plant - ('build' "powr", 300) in the safe west base next to your - Construction Yard BEFORE tick 1500, build a War Factory - ('build' "weap", 2000), then produce three medium tanks - ('build' "2tnk", 850 each). - The deadline is tighter — tick 4500 — so commit the redundant - power plant early; do not wait for the exposed one to fall. If - the strike leaves you with no power, the war factory halves its - output and the army misses the clock. WIN = you brought power - then a war factory online, you still own a Power Plant, you - have 3 medium tanks, and you still own your Construction Yard, - before tick 4500. Stalling, or relying on the single exposed - power plant, misses the bar. + Same base and strike as easy, but a tighter clock — about 50 + turns. End up owning a Power Plant, a War Factory, and three + medium tanks, keeping your Construction Yard. starting_cash: 6000 overrides: actors: @@ -316,21 +331,29 @@ levels: # generalise — the agent must read the actual Construction Yard # latitude and place the redundant power plant beside it. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same N+1 build-order task as medium (one EXPOSED Power Plant + # forward at the east edge that an enemy strike razes at tick + # 1500, $6000, tick 4500 deadline) but your base may begin in + # the NORTH band (y≈12) OR the SOUTH band (y≈26) of the map + # depending on the seed. Read the Construction Yard's actual + # position from the observation and place the redundant Power + # Plant beside it in the safe west base BEFORE tick 1500; build a + # War Factory; then produce three medium tanks. A memorised + # placement cell will mis-place out of build radius on one of the + # two spawns. WIN = you brought power then a war factory online, + # you still own a Power Plant, you have 3 medium tanks, and you + # still own your Construction Yard, before tick 4500. The same + # single-point-of-failure plays — stalling, or relying on the + # lone exposed power plant — lose as on medium. description: > - Same N+1 build-order task as medium (one EXPOSED Power Plant - forward at the east edge that an enemy strike razes at tick - 1500, $6000, tick 4500 deadline) but your base may begin in - the NORTH band (y≈12) OR the SOUTH band (y≈26) of the map - depending on the seed. Read the Construction Yard's actual - position from the observation and place the redundant Power - Plant beside it in the safe west base BEFORE tick 1500; build a - War Factory; then produce three medium tanks. A memorised - placement cell will mis-place out of build radius on one of the - two spawns. WIN = you brought power then a war factory online, - you still own a Power Plant, you have 3 medium tanks, and you - still own your Construction Yard, before tick 4500. The same - single-point-of-failure plays — stalling, or relying on the - lone exposed power plant — lose as on medium. + Same task as medium, but your base sits in the north or south + band depending on seed. End up owning a Power Plant, a War + Factory, and three medium tanks within about 50 turns, keeping + your Construction Yard. starting_cash: 6000 overrides: actors: diff --git a/openra_bench/scenarios/packs/build-tech-skip-decision.yaml b/openra_bench/scenarios/packs/build-tech-skip-decision.yaml index 2d0424ce9b62add0b1fc5a64b31941ba01544376..7732a9616ac55d0f1b8609ae4443d2b6e11c93f1 100644 --- a/openra_bench/scenarios/packs/build-tech-skip-decision.yaml +++ b/openra_bench/scenarios/packs/build-tech-skip-decision.yaml @@ -155,17 +155,24 @@ levels: # as a real reachable LOSS (the old max_turns 25 could hit the turn # cap at tick ~1578 before the deadline → degenerate draw). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Your base already has a Construction Yard (fact) and an Allied + # barracks (tent) — so rifle infantry (e1, $100 each) are + # buildable from turn 1 with NO prior tech step. A light enemy + # garrison of 5 rifle infantry is advancing from the east. The + # objective needs only basic units: train an e1 swarm and rally it + # at your base front to meet the garrison. Do NOT climb the tech + # chain to a war factory and tanks — that is a whole tech tier the + # objective never asked for and it overruns the clock. Win when + # ≥4 enemy units are killed AND your fact still stands, before + # tick 1600. Stalling or teching to tanks both LOSE on the clock. description: > - Your base already has a Construction Yard (fact) and an Allied - barracks (tent) — so rifle infantry (e1, $100 each) are - buildable from turn 1 with NO prior tech step. A light enemy - garrison of 5 rifle infantry is advancing from the east. The - objective needs only basic units: train an e1 swarm and rally it - at your base front to meet the garrison. Do NOT climb the tech - chain to a war factory and tanks — that is a whole tech tier the - objective never asked for and it overruns the clock. Win when - ≥4 enemy units are killed AND your fact still stands, before - tick 1600. Stalling or teching to tanks both LOSE on the clock. + You start with a Construction Yard, a Barracks, and $6000. A + light rifle garrison advances from the east. Kill four enemies + within about 17 turns, keeping your construction yard. starting_cash: 6000 overrides: actors: @@ -199,15 +206,22 @@ levels: # max_turns 30 keeps the loop alive to that terminal frame so the # after_ticks 1401 fail bites as a real reachable LOSS. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same base (fact + tent pre-placed, e1 buildable from turn 1) but + # a larger enemy garrison (7 rifle infantry) and a tighter + # deadline. The objective still needs only basic units: train an + # e1 swarm and rally it at your base front. Climbing the tech + # chain to a war factory and tanks is an unnecessary tier that + # overruns the clock. Win when ≥6 enemy units are killed AND your + # fact still stands, before tick 1400. Stalling or teching to + # tanks both LOSE on the clock. description: > - Same base (fact + tent pre-placed, e1 buildable from turn 1) but - a larger enemy garrison (7 rifle infantry) and a tighter - deadline. The objective still needs only basic units: train an - e1 swarm and rally it at your base front. Climbing the tech - chain to a war factory and tanks is an unnecessary tier that - overruns the clock. Win when ≥6 enemy units are killed AND your - fact still stands, before tick 1400. Stalling or teching to - tanks both LOSE on the clock. + Same base with $6000, but seven rifle attackers and a tighter + clock. Kill six enemies within about 15 turns, keeping your + construction yard. starting_cash: 6000 overrides: actors: @@ -243,15 +257,22 @@ levels: # real reachable LOSS (the old max_turns 23 hit the turn cap at # tick ~1578 before the deadline on seeds 1/3 → degenerate draw). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Same step-pruning call as medium (clear a 7-rifle garrison with + # a basic e1 swarm before tick 1400; skip the unnecessary war- + # factory tech tier) but your base may begin in the NORTH band + # (y≈14) or the SOUTH band (y≈26) depending on the seed. Read + # your barracks' actual position from the observation, train the + # e1 swarm and rally it at your base front. Win when ≥6 enemy + # units are killed AND your fact still stands, before tick 1400. + # Stalling or teching to tanks both LOSE on the clock. description: > - Same step-pruning call as medium (clear a 7-rifle garrison with - a basic e1 swarm before tick 1400; skip the unnecessary war- - factory tech tier) but your base may begin in the NORTH band - (y≈14) or the SOUTH band (y≈26) depending on the seed. Read - your barracks' actual position from the observation, train the - e1 swarm and rally it at your base front. Win when ≥6 enemy - units are killed AND your fact still stands, before tick 1400. - Stalling or teching to tanks both LOSE on the clock. + Same task as medium, but your base sits in the north or south + band depending on seed. Kill six enemies within about 15 turns, + keeping your construction yard. starting_cash: 6000 overrides: actors: diff --git a/openra_bench/scenarios/packs/building-and-planning.yaml b/openra_bench/scenarios/packs/building-and-planning.yaml index f098e7c115203637bb75df3595f6ebc9a0cd0405..3ae78684550520db039205cc01c2ce491a9b487e 100644 --- a/openra_bench/scenarios/packs/building-and-planning.yaml +++ b/openra_bench/scenarios/packs/building-and-planning.yaml @@ -63,12 +63,19 @@ levels: # (no tent); idle/stall loses on the clock. Generous, tick-aligned # clock. max_turns 30 → reachable tick 93+90·29 = 2703. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # No power is pre-placed. Build a power plant first, THEN the + # power-dependent barracks (tent) — a power-less barracks never + # completes. Win = own a 'tent' AND ≥3 buildings total. Building + # the barracks first never completes (no power); spamming power + # plants never gives a barracks; idling loses on the clock. description: > - No power is pre-placed. Build a power plant first, THEN the - power-dependent barracks (tent) — a power-less barracks never - completes. Win = own a 'tent' AND ≥3 buildings total. Building - the barracks first never completes (no power); spamming power - plants never gives a barracks; idling loses on the clock. + No power is pre-placed. End up owning a Barracks (tent) and at + least three buildings total within about 27 turns. The barracks + requires power to complete. starting_cash: 6000 win_condition: all_of: @@ -88,13 +95,20 @@ levels: # loses (not in region); idle loses on the clock. # max_turns 40 → reachable tick 93+90·39 = 3603. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Build a defensive line to the EAST: stand up power, then the + # barracks it enables, then place at least two pillboxes (pbox — + # they need the barracks, which needs power) INSIDE the designated + # eastern region near (40,20). Skipping the chain never yields a + # pillbox; placing the pillboxes near the base (wrong direction) + # does not count; idling loses on the clock. description: > - Build a defensive line to the EAST: stand up power, then the - barracks it enables, then place at least two pillboxes (pbox — - they need the barracks, which needs power) INSIDE the designated - eastern region near (40,20). Skipping the chain never yields a - pillbox; placing the pillboxes near the base (wrong direction) - does not count; idling loses on the clock. + Place at least two pillboxes within nine cells of (40,20) — the + eastern region — within about 38 turns. Pillboxes need a + barracks, which needs power. starting_cash: 5000 win_condition: all_of: @@ -114,14 +128,21 @@ levels: # creep that omits the tech chain never yields a pillbox; idle loses # on the clock. max_turns 70 → reachable tick 93+90·69 = 6303. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Found the defensive line FAR to the east, near (60,20) — beyond + # the construction yard's reach. Creep the base east (each new + # power plant extends where you may build) until you can stand up + # the barracks and place at least two pillboxes (pbox → tent → + # power chain) inside the far region, before the tight deadline. + # Building near the spawn never reaches the region; a power creep + # that skips the barracks never yields a pillbox; idling loses. description: > - Found the defensive line FAR to the east, near (60,20) — beyond - the construction yard's reach. Creep the base east (each new - power plant extends where you may build) until you can stand up - the barracks and place at least two pillboxes (pbox → tent → - power chain) inside the far region, before the tight deadline. - Building near the spawn never reaches the region; a power creep - that skips the barracks never yields a pillbox; idling loses. + Place at least two pillboxes within ten cells of (60,20) — far + beyond your construction yard's reach — within about 64 turns. + Each new building extends where you may build next. starting_cash: 6000 win_condition: all_of: diff --git a/openra_bench/scenarios/packs/combat-attack-from-behind-fog.yaml b/openra_bench/scenarios/packs/combat-attack-from-behind-fog.yaml index ca049125a67dcc0742d568bd8e8471c6ebf6be28..4a7f14d33d583c3a9ad16cf0f8a44c0f10dfb1ec 100644 --- a/openra_bench/scenarios/packs/combat-attack-from-behind-fog.yaml +++ b/openra_bench/scenarios/packs/combat-attack-from-behind-fog.yaml @@ -164,22 +164,30 @@ levels: # engaging the line). Stall LOSES (line is stance:2, never # advances; fact never razed). Fog flank cleanly WINS. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). + # A LIGHT vertical line of THREE anti-tank rocket soldiers (e3, + # Dragon launcher, range ~5) stands at x=50, y=19..21, facing + # west (stance:2 — auto-fire in range, no chase). Behind the + # line, an UNDEFENDED enemy construction yard (fact) sits at + # (100,20). Charging head-on (east along y=20) puts the lead + # tank in range of all three e3 and the column burns turns + # reducing the line — the fact at x=100 is still 50 cells away + # with the deadline closing. The winning play is the FOG FLANK + # — route the strike force to the far north (y=2) or far south + # (y=38), well outside the line's range, drive east past x=80, + # then turn inward to descend on the fact at (100,20). Win + # when the fact at (100,20) is destroyed AND ≥2 of your tanks + # survive, before tick 5400. Stalling LOSES (fact never + # destroyed); frontal-charge LOSES on the clock. description: > - Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). - A LIGHT vertical line of THREE anti-tank rocket soldiers (e3, - Dragon launcher, range ~5) stands at x=50, y=19..21, facing - west (stance:2 — auto-fire in range, no chase). Behind the - line, an UNDEFENDED enemy construction yard (fact) sits at - (100,20). Charging head-on (east along y=20) puts the lead - tank in range of all three e3 and the column burns turns - reducing the line — the fact at x=100 is still 50 cells away - with the deadline closing. The winning play is the FOG FLANK - — route the strike force to the far north (y=2) or far south - (y=38), well outside the line's range, drive east past x=80, - then turn inward to descend on the fact at (100,20). Win - when the fact at (100,20) is destroyed AND ≥2 of your tanks - survive, before tick 5400. Stalling LOSES (fact never - destroyed); frontal-charge LOSES on the clock. + Four tanks stage at the west edge. A light rocket-infantry line + at x=50 blocks the centre; the enemy construction yard at + (100,20) is undefended. Destroy the yard with at least two tanks + surviving, within about 65 turns. overrides: actors: # STRIKE FORCE — 4 medium tanks stacked at the west edge, @@ -222,23 +230,31 @@ levels: # wins comfortably: the line never fires on the flanker (Manhattan # ≥13 to the nearest defender from y=2 or y=38). medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). + # A HEAVY defensive line stands at x=50, y=15..25, facing west + # (stance:2 — auto-fire in range, no chase): TWO pillboxes + # (pbox, range ~6) bookend FIVE anti-tank rocket soldiers (e3, + # Dragon range ~5). Behind the line, an UNDEFENDED enemy + # construction yard (fact) sits at (100,20). Charging head-on + # (east along y=18..22) puts the lead tank inside the + # overlapping kill envelopes of 4+ defenders simultaneously; + # concentrated fire destroys the column before it clears. + # The winning play is the FOG FLANK — route the strike force + # WELL off-axis to the far north (y=2) or far south (y=38), + # drive east past the line's longitude (x=50), then turn + # inward to descend on the fact at (100,20) from behind. Win + # when the fact at (100,20) is destroyed AND ≥2 of your tanks + # survive, before tick 4500. Stall, brute attack_move east, + # and any frontal charge all LOSE. description: > - Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). - A HEAVY defensive line stands at x=50, y=15..25, facing west - (stance:2 — auto-fire in range, no chase): TWO pillboxes - (pbox, range ~6) bookend FIVE anti-tank rocket soldiers (e3, - Dragon range ~5). Behind the line, an UNDEFENDED enemy - construction yard (fact) sits at (100,20). Charging head-on - (east along y=18..22) puts the lead tank inside the - overlapping kill envelopes of 4+ defenders simultaneously; - concentrated fire destroys the column before it clears. - The winning play is the FOG FLANK — route the strike force - WELL off-axis to the far north (y=2) or far south (y=38), - drive east past the line's longitude (x=50), then turn - inward to descend on the fact at (100,20) from behind. Win - when the fact at (100,20) is destroyed AND ≥2 of your tanks - survive, before tick 4500. Stall, brute attack_move east, - and any frontal charge all LOSE. + Heavier line at x=50: two pillboxes plus five rocket soldiers + spanning y=15 to 25. The enemy construction yard at (100,20) + remains undefended. Destroy the yard with at least two tanks + surviving, within about 54 turns. overrides: actors: - {type: 2tnk, owner: agent, position: [6, 18], stance: 1} @@ -293,24 +309,33 @@ levels: # fog-flank decision (NORTH spawn → fog via y=2; SOUTH spawn # → fog via y=38). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at ONE of two west-edge + # corridors (NORTH y=14..17 OR SOUTH y=23..26, chosen by seed, + # anti-memorisation). A dense vertical wall stands at x=50, + # y=15..25, facing west (stance:2 — auto-fire in range, no + # chase): THREE pillboxes (pbox) interleaved with SEVEN anti- + # tank rocket soldiers (e3, Dragon launcher, range ~5). Behind + # the wall, an UNDEFENDED enemy construction yard (fact) sits + # at (100,20). Any frontal charge along y=14..26 enters the + # overlapping kill envelope of the entire 10-defender wall; + # concentrated fire destroys the column before it clears. + # The winning play is the FOG FLANK — route the strike force + # to the far north (y=2 from the NORTH spawn) or far south + # (y=38 from the SOUTH spawn), drive east past x=80, then + # turn inward to descend on the fact at (100,20). Win when + # the fact at (100,20) is destroyed AND ≥2 of your tanks + # survive, before tick 4500. Stall, brute attack_move east, + # and any frontal head-on charge all LOSE. description: > - Four medium tanks (2tnk) stage at ONE of two west-edge - corridors (NORTH y=14..17 OR SOUTH y=23..26, chosen by seed, - anti-memorisation). A dense vertical wall stands at x=50, - y=15..25, facing west (stance:2 — auto-fire in range, no - chase): THREE pillboxes (pbox) interleaved with SEVEN anti- - tank rocket soldiers (e3, Dragon launcher, range ~5). Behind - the wall, an UNDEFENDED enemy construction yard (fact) sits - at (100,20). Any frontal charge along y=14..26 enters the - overlapping kill envelope of the entire 10-defender wall; - concentrated fire destroys the column before it clears. - The winning play is the FOG FLANK — route the strike force - to the far north (y=2 from the NORTH spawn) or far south - (y=38 from the SOUTH spawn), drive east past x=80, then - turn inward to descend on the fact at (100,20). Win when - the fact at (100,20) is destroyed AND ≥2 of your tanks - survive, before tick 4500. Stall, brute attack_move east, - and any frontal head-on charge all LOSE. + Same task as medium with a denser wall (three pillboxes plus + seven rocket soldiers at x=50). Your tanks stage in the north + or south corridor depending on seed. Destroy the enemy yard at + (100,20) with at least two tanks surviving, within about 54 + turns. overrides: actors: # spawn_point 0 — NORTH staging corridor (y=14..17). Fog diff --git a/openra_bench/scenarios/packs/combat-bait-counter-attack.yaml b/openra_bench/scenarios/packs/combat-bait-counter-attack.yaml index 33a53f358ac63c5ad6cc1c20a6849f0f669bf982..0e55e8e54c21e2bba6a1b618d5773819fc14412e 100644 --- a/openra_bench/scenarios/packs/combat-bait-counter-attack.yaml +++ b/openra_bench/scenarios/packs/combat-bait-counter-attack.yaml @@ -111,19 +111,26 @@ levels: # them at their leash the strike tanks focus-fire the nearest guard # and then drive onto the now-undefended yard. easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Three leashed rocket-infantry guards stand on the west face of + # the enemy construction yard (`fact` at 80,20), directly + # between you and the objective. They hold post, auto-fire on + # units in range, lunge at the nearest foe within ≈16 cells, and + # snap back past a leash of ≈18. Destroy the enemy construction + # yard with at most 3 of your units lost before tick 5400. + # Driving the tanks straight east at the yard puts them in + # anti-tank fire from the whole guard line; stalling runs the + # clock. Commit the fast jeep on a divergent south-east vector + # so the guards lunge south after it; while the jeep kites them + # at their leash, run the strike tanks straight through the + # now-undefended west face to destroy the yard. description: > - Three leashed rocket-infantry guards stand on the west face of - the enemy construction yard (`fact` at 80,20), directly - between you and the objective. They hold post, auto-fire on - units in range, lunge at the nearest foe within ≈16 cells, and - snap back past a leash of ≈18. Destroy the enemy construction - yard with at most 3 of your units lost before tick 5400. - Driving the tanks straight east at the yard puts them in - anti-tank fire from the whole guard line; stalling runs the - clock. Commit the fast jeep on a divergent south-east vector - so the guards lunge south after it; while the jeep kites them - at their leash, run the strike tanks straight through the - now-undefended west face to destroy the yard. + Three leashed rocket-infantry guards wall the west face of the + enemy construction yard at (80,20). Destroy the yard, losing at + most three units, within about 62 turns. overrides: actors: # Strike force: 4 medium tanks at west, central staging. @@ -175,19 +182,26 @@ levels: # jeep + at most one tank): the strike must still hit the yard # essentially intact; burning a second tank fails. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # A denser cluster of 5 leashed rocket-infantry guards covers + # the enemy construction yard (`fact` at 80,20) on the WEST and + # SOUTH faces (the two natural approaches from your west base). + # A head-on push without first thinning the cover is decisively + # fatal. You command 5 medium tanks staged on the NORTH flank + # plus the bait jeep, but may spend at most 2 units — losing a + # second tank fails. Commit the fast jeep on a deep south flank + # vector to pull the south arc of guards off post after it; while + # that arc is displaced, swing the strike tanks around the + # now-vacated NORTH flank and onto the yard before tick 5400. + # Bait-only (no strike) loses on the clock; brute frontal trades + # the strike force. description: > - A denser cluster of 5 leashed rocket-infantry guards covers - the enemy construction yard (`fact` at 80,20) on the WEST and - SOUTH faces (the two natural approaches from your west base). - A head-on push without first thinning the cover is decisively - fatal. You command 5 medium tanks staged on the NORTH flank - plus the bait jeep, but may spend at most 2 units — losing a - second tank fails. Commit the fast jeep on a deep south flank - vector to pull the south arc of guards off post after it; while - that arc is displaced, swing the strike tanks around the - now-vacated NORTH flank and onto the yard before tick 5400. - Bait-only (no strike) loses on the clock; brute frontal trades - the strike force. + Denser cover — five leashed rocket-infantry guards on the west + and south faces of the enemy yard at (80,20). Destroy the yard, + losing at most two units, within about 62 turns. overrides: actors: # Strike force: 5 medium tanks staged on the NORTH flank @@ -234,20 +248,27 @@ levels: # closer guard arc engages the column before the bait can pull it). # Same 5-guard L cover, 5-tank strike, and loss cap 2 as medium. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # The same dense 5-guard L-cover of the enemy construction yard + # (`fact` at 80,20) as the previous tier, but your force stages + # from a seed-chosen distance — either the far west or well + # forward — so a memorised fixed opening cannot generalise; read + # your tanks' actual position from the observation. You command 5 + # medium tanks staged on the NORTH flank plus the bait jeep, but + # may spend at most 2 units (the jeep + at most one tank) — + # losing a second tank fails. Commit the fast jeep on a deep + # south flank vector to pull the south arc of guards off post; + # while that arc is displaced, swing the strike tanks around the + # now-vacated NORTH flank and onto the yard before tick 5400. + # Brute frontal trades armour and fails; bait-only never razes + # the yard; stalling loses the clock. description: > - The same dense 5-guard L-cover of the enemy construction yard - (`fact` at 80,20) as the previous tier, but your force stages - from a seed-chosen distance — either the far west or well - forward — so a memorised fixed opening cannot generalise; read - your tanks' actual position from the observation. You command 5 - medium tanks staged on the NORTH flank plus the bait jeep, but - may spend at most 2 units (the jeep + at most one tank) — - losing a second tank fails. Commit the fast jeep on a deep - south flank vector to pull the south arc of guards off post; - while that arc is displaced, swing the strike tanks around the - now-vacated NORTH flank and onto the yard before tick 5400. - Brute frontal trades armour and fails; bait-only never razes - the yard; stalling loses the clock. + Same task as medium, but your force stages from a seed-chosen + distance (far west or well forward). Destroy the enemy yard, + losing at most two units, within about 62 turns. overrides: actors: # spawn_point 0 — FAR-WEST staging (x=6): the strike has the diff --git a/openra_bench/scenarios/packs/combat-divide-and-conquer.yaml b/openra_bench/scenarios/packs/combat-divide-and-conquer.yaml index f55908a13f7e51ce7cab8180a19c4ddde89eeeae..deff4b223ec71d23249f3fb2cb7f314565a0bb9c 100644 --- a/openra_bench/scenarios/packs/combat-divide-and-conquer.yaml +++ b/openra_bench/scenarios/packs/combat-divide-and-conquer.yaml @@ -142,18 +142,26 @@ levels: # inert-easy-teeth convention — the divide-vs-converge delta lives # on medium and hard). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You command 4 medium tanks (2tnk) staging at the west edge + # (x=6, y=18..21). Two enemy clusters sit at x=60: a NORTH + # cluster at y=15 (2× e3 anti-tank rocketeers, range ~5) and + # a SOUTH cluster at y=25 (same). The clusters are 10 cells + # apart — engaging on the y=20 midpoint puts you in range of + # both at once. The winning play is divide-and-conquer: flank + # one cluster (e.g. via y=5 or y=35), eliminate it, then pivot + # to the other. Your construction yard (fact) sits at the deep + # rear (4,20); it must survive. Win when 4 enemies are killed, + # at least 3 of your tanks remain, your fact still stands, and + # the deadline (tick 4500) has not passed. description: > - You command 4 medium tanks (2tnk) staging at the west edge - (x=6, y=18..21). Two enemy clusters sit at x=60: a NORTH - cluster at y=15 (2× e3 anti-tank rocketeers, range ~5) and - a SOUTH cluster at y=25 (same). The clusters are 10 cells - apart — engaging on the y=20 midpoint puts you in range of - both at once. The winning play is divide-and-conquer: flank - one cluster (e.g. via y=5 or y=35), eliminate it, then pivot - to the other. Your construction yard (fact) sits at the deep - rear (4,20); it must survive. Win when 4 enemies are killed, - at least 3 of your tanks remain, your fact still stands, and - the deadline (tick 4500) has not passed. + Four tanks at the west edge. Two enemy rocket-infantry clusters + sit at x=60, ten cells apart at y=15 and y=25. Kill four + enemies, keep at least three tanks and your construction yard, + within about 50 turns. overrides: actors: # STRIKE FORCE — 4 medium tanks at the west edge, stance:1 @@ -206,19 +214,26 @@ levels: # one cluster at a time in clean 1-vs-1-cluster geometry, preserving # all 4 tanks. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You command 4 medium tanks (2tnk) staging at the west edge + # (x=6, y=18..21). Two HEAVY enemy clusters sit at x=60: a NORTH + # cluster at y=15 (3× e3 anti-tank rocketeers + 1× 1tnk light + # tank) and a SOUTH cluster at y=25 (same). The clusters are + # 10 cells apart — engaging on the y=20 midpoint puts you inside + # weapon range of BOTH clusters at once (8 enemies firing on the + # lead). The winning play is divide-and-conquer: flank well + # NORTH (e.g. y=5) so only Cluster A is in range, eliminate it, + # then pivot SOUTH (e.g. y=35) and engage Cluster B in isolation. + # Your construction yard (fact) at (4,20) must survive. Win when + # 8 enemies are killed, at least 3 of your tanks remain, your + # fact still stands, and the deadline (tick 4500) has not passed. description: > - You command 4 medium tanks (2tnk) staging at the west edge - (x=6, y=18..21). Two HEAVY enemy clusters sit at x=60: a NORTH - cluster at y=15 (3× e3 anti-tank rocketeers + 1× 1tnk light - tank) and a SOUTH cluster at y=25 (same). The clusters are - 10 cells apart — engaging on the y=20 midpoint puts you inside - weapon range of BOTH clusters at once (8 enemies firing on the - lead). The winning play is divide-and-conquer: flank well - NORTH (e.g. y=5) so only Cluster A is in range, eliminate it, - then pivot SOUTH (e.g. y=35) and engage Cluster B in isolation. - Your construction yard (fact) at (4,20) must survive. Win when - 8 enemies are killed, at least 3 of your tanks remain, your - fact still stands, and the deadline (tick 4500) has not passed. + Heavier clusters at y=15 and y=25 (rocket soldiers plus a light + tank each, eight enemies total). Kill all eight, keep at least + three tanks and your construction yard, within about 50 turns. overrides: actors: - {type: 2tnk, owner: agent, position: [6, 18], stance: 1} @@ -269,20 +284,28 @@ levels: # The survival cap also tightens to own_units_gte:3 (only 1 tank # may be lost across the whole campaign on hard). hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # You command 4 medium tanks (2tnk) staging at ONE of two west- + # edge corridors (NORTH y=10..13 or SOUTH y=27..30, chosen by + # seed). Two HEAVY enemy clusters sit at x=60: a NORTH cluster + # at y=15 (3× e3 anti-tank rocketeers + 1× 1tnk) and a SOUTH + # cluster at y=25 (same). The clusters are 10 cells apart — + # engaging on the y=20 midpoint puts you inside weapon range of + # BOTH clusters at once (8 enemies firing on the lead). The + # winning play is divide-and-conquer: flank well NORTH (y=5) or + # SOUTH (y=35) so only ONE cluster is in range, eliminate it, + # then pivot to the OTHER side and engage the second cluster + # in isolation. Your construction yard (fact) at (4,20) must + # survive. Win when 8 enemies are killed, at least 3 of your + # tanks remain, your fact still stands, before tick 4500. description: > - You command 4 medium tanks (2tnk) staging at ONE of two west- - edge corridors (NORTH y=10..13 or SOUTH y=27..30, chosen by - seed). Two HEAVY enemy clusters sit at x=60: a NORTH cluster - at y=15 (3× e3 anti-tank rocketeers + 1× 1tnk) and a SOUTH - cluster at y=25 (same). The clusters are 10 cells apart — - engaging on the y=20 midpoint puts you inside weapon range of - BOTH clusters at once (8 enemies firing on the lead). The - winning play is divide-and-conquer: flank well NORTH (y=5) or - SOUTH (y=35) so only ONE cluster is in range, eliminate it, - then pivot to the OTHER side and engage the second cluster - in isolation. Your construction yard (fact) at (4,20) must - survive. Win when 8 enemies are killed, at least 3 of your - tanks remain, your fact still stands, before tick 4500. + Same heavy clusters as medium, but your tanks stage in the north + or south corridor depending on seed. Kill all eight enemies, keep + at least three tanks and your construction yard, within about + 50 turns. overrides: actors: # spawn_point 0 — NORTH staging corridor y=10..13. Closer diff --git a/openra_bench/scenarios/packs/combat-flanking-attack.yaml b/openra_bench/scenarios/packs/combat-flanking-attack.yaml index 419d6ed656398937cae3ad9fe42b75fc6e4a9491..bca8f42937e8f7f0cf8013d413695d1f233892ee 100644 --- a/openra_bench/scenarios/packs/combat-flanking-attack.yaml +++ b/openra_bench/scenarios/packs/combat-flanking-attack.yaml @@ -130,20 +130,27 @@ levels: # the lead, just under one-shot range). Stall LOSES on the clock # (defenders are stance:2 ReturnFire-equivalent, never advance). easy: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). + # A short vertical line of THREE anti-tank rocket soldiers (e3, + # Dragon launcher, range ~5) stands at x=60, y=19..21, facing + # west (stance:2 — auto-fire in range, no chase). Charging + # head-on (east along y=20) puts the lead tank in range of all + # three rocket soldiers at once. The winning play is the FLANK + # — move the strike force off-axis (north of y=18 or south of + # y=22) and approach the line end-on so only 1-2 rocket + # soldiers are in range at any moment. Win when 3 enemy units + # are killed AND at least 3 of your tanks remain, before tick + # 4500. Stalling LOSES (kill bar unmet). Brute attack_move east + # can squeak the survival bar on easy — the bigger tests are + # on medium and hard. description: > - Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). - A short vertical line of THREE anti-tank rocket soldiers (e3, - Dragon launcher, range ~5) stands at x=60, y=19..21, facing - west (stance:2 — auto-fire in range, no chase). Charging - head-on (east along y=20) puts the lead tank in range of all - three rocket soldiers at once. The winning play is the FLANK - — move the strike force off-axis (north of y=18 or south of - y=22) and approach the line end-on so only 1-2 rocket - soldiers are in range at any moment. Win when 3 enemy units - are killed AND at least 3 of your tanks remain, before tick - 4500. Stalling LOSES (kill bar unmet). Brute attack_move east - can squeak the survival bar on easy — the bigger tests are - on medium and hard. + Four tanks at the west edge face a short vertical line of three + rocket soldiers at x=60. Kill three enemies, keep at least three + tanks, within about 50 turns. overrides: actors: # STRIKE FORCE — 4 medium tanks stacked at the west edge, @@ -183,21 +190,28 @@ levels: # all 4 tanks because only 1-2 e3 are ever in range of the leading # flanker at a time. medium: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). + # A tight vertical line of FIVE anti-tank rocket soldiers (e3, + # Dragon launcher, range ~5) stands at x=60, y=18..22, facing + # west (stance:2 — auto-fire in range, no chase). Charging + # head-on along the engagement axis (y=20) puts the lead tank + # inside range of ALL FIVE rocket soldiers simultaneously — + # concentrated fire pressures the column. The winning play is + # the FLANK — move the strike force off-axis (well NORTH of + # y=18 or well SOUTH of y=22, e.g. y=8 or y=32) and approach + # the line END-ON so only one or two rocket soldiers are in + # Dragon range of the leading flanker at any moment; pick them + # off in sequence. Win when 4 enemy units are killed AND at + # least 3 of your tanks remain, before tick 4500. Stall and + # brute attack_move east both LOSE. description: > - Four medium tanks (2tnk) stage at the west edge (x=6, y=18..21). - A tight vertical line of FIVE anti-tank rocket soldiers (e3, - Dragon launcher, range ~5) stands at x=60, y=18..22, facing - west (stance:2 — auto-fire in range, no chase). Charging - head-on along the engagement axis (y=20) puts the lead tank - inside range of ALL FIVE rocket soldiers simultaneously — - concentrated fire pressures the column. The winning play is - the FLANK — move the strike force off-axis (well NORTH of - y=18 or well SOUTH of y=22, e.g. y=8 or y=32) and approach - the line END-ON so only one or two rocket soldiers are in - Dragon range of the leading flanker at any moment; pick them - off in sequence. Win when 4 enemy units are killed AND at - least 3 of your tanks remain, before tick 4500. Stall and - brute attack_move east both LOSE. + Tighter five-rocket-soldier line at x=60 spanning y=18 to 22. + Kill four enemies, keep at least three tanks, within about 50 + turns. overrides: actors: - {type: 2tnk, owner: agent, position: [6, 18], stance: 1} @@ -241,24 +255,31 @@ levels: # symmetric across the map's mid-latitude (y=20) so either # spawn faces an equivalent flank decision. hard: + # Original (pre-verbosity-sweep) description preserved for + # contributors. The trimmed version below removes scripted-policy + # spoilers and cell-coord dumps; load-bearing intent kept. + # + # Four medium tanks (2tnk) stage at ONE of two west-edge + # corridors (NORTH y=14..17 OR SOUTH y=23..26, chosen by seed, + # anti-memorisation). A two-column wall of anti-tank rocket + # soldiers (e3, Dragon launcher, range ~5) — SIX in the front + # rank at x=60 (y=17..22) and FOUR in a back rank at x=62 — + # stands facing west (stance:2 — auto-fire in range, no + # chase). Charging east along any engagement axis between y=14 + # and y=26 puts the lead tank inside Dragon range of MOST of + # the 10-e3 wall ⇒ concentrated rocket fire destroys ≥1 tank + # before the column clears the line. The winning play is the + # FLANK — approach from WELL off-axis (y=8 from the north + # spawn or y=32 from the south spawn) and engage the line + # END-ON so only one or two rocket soldiers are in range of + # the leading flanker at any moment. Win when 4 enemy units + # are killed AND ALL FOUR of your tanks remain, before tick + # 4500. Stall, brute attack_move east, and any frontal head-on + # charge all LOSE. description: > - Four medium tanks (2tnk) stage at ONE of two west-edge - corridors (NORTH y=14..17 OR SOUTH y=23..26, chosen by seed, - anti-memorisation). A two-column wall of anti-tank rocket - soldiers (e3, Dragon launcher, range ~5) — SIX in the front - rank at x=60 (y=17..22) and FOUR in a back rank at x=62 — - stands facing west (stance:2 — auto-fire in range, no - chase). Charging east along any engagement axis between y=14 - and y=26 puts the lead tank inside Dragon range of MOST of - the 10-e3 wall ⇒ concentrated rocket fire destroys ≥1 tank - before the column clears the line. The winning play is the - FLANK — approach from WELL off-axis (y=8 from the north - spawn or y=32 from the south spawn) and engage the line - END-ON so only one or two rocket soldiers are in range of - the leading flanker at any moment. Win when 4 enemy units - are killed AND ALL FOUR of your tanks remain, before tick - 4500. Stall, brute attack_move east, and any frontal head-on - charge all LOSE. + Two-column rocket-soldier wall at x=60 (ten enemies total). Tanks + stage in the north or south corridor depending on seed. Kill + five enemies, keep at least three tanks, within about 50 turns. overrides: actors: # spawn_point 0 — NORTH staging corridor (y=14..17). Closer diff --git a/openra_bench/scenarios/packs/def-bridge-chokepoint.yaml b/openra_bench/scenarios/packs/def-bridge-chokepoint.yaml new file mode 100644 index 0000000000000000000000000000000000000000..9af240b2b2e6214a27197473673ed57ef3d0da5f --- /dev/null +++ b/openra_bench/scenarios/packs/def-bridge-chokepoint.yaml @@ -0,0 +1,2046 @@ +# def-bridge-chokepoint — chokepoint defense at narrow bridges. +# Generated; do not edit the actors/water_cells blocks by hand. +# +# Design: medium + hard each place ONE proc directly behind each bridge +# as the per-bridge target. An undefended bridge = its proc falls = LOSS +# (the win predicate requires ALL procs alive). Easy uses single-fact +# target because there's only one bridge. +# +# ⚠ Status (2026-05-23): +# easy tier: scripted-policy bar HOLDS clean — stall LOSS, intended +# (set_stance + attack_move to bridge) WIN. +# medium tier: bar holds — stall LOSS, only-centre LOSS (one proc +# falls), split-all WIN. +# hard tier: only-centre still WRONGLY wins (kill=16, all procs +# alive). Root cause: enemy stance:3 (AttackAnything) +# prefers fighting visible UNITS over buildings, so +# attackers from the outer bridges converge on the +# central defender cluster instead of marching to their +# designated proc. The defender column en route to the +# centre intercepts them all. FIX (engine, Phase 1): +# add `ScriptedBehavior::BuildingRusher` (variant of +# Rusher that explicitly attack-orders the nearest +# AGENT BUILDING per unit, not the nearest agent unit), +# then switch this pack's `enemy: {bot_type: +# building_rusher}` for medium + hard. Until then, hard +# tier should not be promoted to the active eval suite. +meta: + id: def-bridge-chokepoint + title: Bridge Chokepoint — defend every crossing before they reach you + capability: action + real_world_meaning: Geographic chokepoint defense. A barrier (water) channels every attacker through a small number of crossings + (bridges). The defender must (a) flip the engagement stance so units fire, AND (b) distribute forces across every crossing + — concentrating on one bridge leaves the others wide open and their assets fall. + robotics_analogue: Perimeter defense around a facility with a moat and discrete gates. Guards posted on standby must be + told to engage AND stationed at EACH gate — an undefended gate is a free corridor for the intruder to reach the nearest + asset. + benchmark_anchor: + - ERQA spatial commit (read terrain → assign defenders to chokepoints) + - MicroRTS chokepoint defense + - 'military doctrine: defense at obstacle-channelized terrain' + author: openra-bench +base_map: rush-hour-arena +starting_cash: 1000 +base: + agent: + faction: allies + cash: 1000 + enemy: + faction: soviet + cash: 0 + tools: + - observe + - move_units + - attack_unit + - attack_move + - stop + - set_stance + spawn_mcvs: false + planning: true + termination: + max_ticks: 6000 + actors: [] +levels: + easy: + description: A water band cuts the map east-to-west with a single bridge at the centre. Six riflemen attack from the north. + Your defenders guard your base on HoldFire (stance 0) — they will NOT engage unless you flip stance and move them to + the bridge. Win = kill ≥4 attackers AND keep your fact alive, before turn 30. + starting_cash: 1000 + overrides: + actors: + - &id001 + type: fact + owner: agent + position: + - 10 + - 33 + - &id002 + type: powr + owner: agent + position: + - 10 + - 35 + - type: e1 + owner: agent + position: + - 58 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 59 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 62 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 63 + - 30 + stance: 0 + - type: e1 + owner: enemy + position: + - 58 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 59 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 60 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 61 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 62 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 63 + - 4 + stance: 3 + - type: fact + owner: enemy + position: + - 120 + - 4 + water_cells: + - [0, 18] + - [1, 18] + - [2, 18] + - [3, 18] + - [4, 18] + - [5, 18] + - [6, 18] + - [7, 18] + - [8, 18] + - [9, 18] + - [10, 18] + - [11, 18] + - [12, 18] + - [13, 18] + - [14, 18] + - [15, 18] + - [16, 18] + - [17, 18] + - [18, 18] + - [19, 18] + - [20, 18] + - [21, 18] + - [22, 18] + - [23, 18] + - [24, 18] + - [25, 18] + - [26, 18] + - [27, 18] + - [28, 18] + - [29, 18] + - [30, 18] + - [31, 18] + - [32, 18] + - [33, 18] + - [34, 18] + - [35, 18] + - [36, 18] + - [37, 18] + - [38, 18] + - [39, 18] + - [40, 18] + - [41, 18] + - [42, 18] + - [43, 18] + - [44, 18] + - [45, 18] + - [46, 18] + - [47, 18] + - [48, 18] + - [49, 18] + - [50, 18] + - [51, 18] + - [52, 18] + - [53, 18] + - [54, 18] + - [55, 18] + - [56, 18] + - [57, 18] + - [58, 18] + - [59, 18] + - [64, 18] + - [65, 18] + - [66, 18] + - [67, 18] + - [68, 18] + - [69, 18] + - [70, 18] + - [71, 18] + - [72, 18] + - [73, 18] + - [74, 18] + - [75, 18] + - [76, 18] + - [77, 18] + - [78, 18] + - [79, 18] + - [80, 18] + - [81, 18] + - [82, 18] + - [83, 18] + - [84, 18] + - [85, 18] + - [86, 18] + - [87, 18] + - [88, 18] + - [89, 18] + - [90, 18] + - [91, 18] + - [92, 18] + - [93, 18] + - [94, 18] + - [95, 18] + - [96, 18] + - [97, 18] + - [98, 18] + - [99, 18] + - [100, 18] + - [101, 18] + - [102, 18] + - [103, 18] + - [104, 18] + - [105, 18] + - [106, 18] + - [107, 18] + - [108, 18] + - [109, 18] + - [110, 18] + - [111, 18] + - [112, 18] + - [113, 18] + - [114, 18] + - [115, 18] + - [116, 18] + - [117, 18] + - [118, 18] + - [119, 18] + - [120, 18] + - [121, 18] + - [122, 18] + - [123, 18] + - [124, 18] + - [125, 18] + - [126, 18] + - [127, 18] + - [0, 19] + - [1, 19] + - [2, 19] + - [3, 19] + - [4, 19] + - [5, 19] + - [6, 19] + - [7, 19] + - [8, 19] + - [9, 19] + - [10, 19] + - [11, 19] + - [12, 19] + - [13, 19] + - [14, 19] + - [15, 19] + - [16, 19] + - [17, 19] + - [18, 19] + - [19, 19] + - [20, 19] + - [21, 19] + - [22, 19] + - [23, 19] + - [24, 19] + - [25, 19] + - [26, 19] + - [27, 19] + - [28, 19] + - [29, 19] + - [30, 19] + - [31, 19] + - [32, 19] + - [33, 19] + - [34, 19] + - [35, 19] + - [36, 19] + - [37, 19] + - [38, 19] + - [39, 19] + - [40, 19] + - [41, 19] + - [42, 19] + - [43, 19] + - [44, 19] + - [45, 19] + - [46, 19] + - [47, 19] + - [48, 19] + - [49, 19] + - [50, 19] + - [51, 19] + - [52, 19] + - [53, 19] + - [54, 19] + - [55, 19] + - [56, 19] + - [57, 19] + - [58, 19] + - [59, 19] + - [64, 19] + - [65, 19] + - [66, 19] + - [67, 19] + - [68, 19] + - [69, 19] + - [70, 19] + - [71, 19] + - [72, 19] + - [73, 19] + - [74, 19] + - [75, 19] + - [76, 19] + - [77, 19] + - [78, 19] + - [79, 19] + - [80, 19] + - [81, 19] + - [82, 19] + - [83, 19] + - [84, 19] + - [85, 19] + - [86, 19] + - [87, 19] + - [88, 19] + - [89, 19] + - [90, 19] + - [91, 19] + - [92, 19] + - [93, 19] + - [94, 19] + - [95, 19] + - [96, 19] + - [97, 19] + - [98, 19] + - [99, 19] + - [100, 19] + - [101, 19] + - [102, 19] + - [103, 19] + - [104, 19] + - [105, 19] + - [106, 19] + - [107, 19] + - [108, 19] + - [109, 19] + - [110, 19] + - [111, 19] + - [112, 19] + - [113, 19] + - [114, 19] + - [115, 19] + - [116, 19] + - [117, 19] + - [118, 19] + - [119, 19] + - [120, 19] + - [121, 19] + - [122, 19] + - [123, 19] + - [124, 19] + - [125, 19] + - [126, 19] + - [127, 19] + - [0, 20] + - [1, 20] + - [2, 20] + - [3, 20] + - [4, 20] + - [5, 20] + - [6, 20] + - [7, 20] + - [8, 20] + - [9, 20] + - [10, 20] + - [11, 20] + - [12, 20] + - [13, 20] + - [14, 20] + - [15, 20] + - [16, 20] + - [17, 20] + - [18, 20] + - [19, 20] + - [20, 20] + - [21, 20] + - [22, 20] + - [23, 20] + - [24, 20] + - [25, 20] + - [26, 20] + - [27, 20] + - [28, 20] + - [29, 20] + - [30, 20] + - [31, 20] + - [32, 20] + - [33, 20] + - [34, 20] + - [35, 20] + - [36, 20] + - [37, 20] + - [38, 20] + - [39, 20] + - [40, 20] + - [41, 20] + - [42, 20] + - [43, 20] + - [44, 20] + - [45, 20] + - [46, 20] + - [47, 20] + - [48, 20] + - [49, 20] + - [50, 20] + - [51, 20] + - [52, 20] + - [53, 20] + - [54, 20] + - [55, 20] + - [56, 20] + - [57, 20] + - [58, 20] + - [59, 20] + - [64, 20] + - [65, 20] + - [66, 20] + - [67, 20] + - [68, 20] + - [69, 20] + - [70, 20] + - [71, 20] + - [72, 20] + - [73, 20] + - [74, 20] + - [75, 20] + - [76, 20] + - [77, 20] + - [78, 20] + - [79, 20] + - [80, 20] + - [81, 20] + - [82, 20] + - [83, 20] + - [84, 20] + - [85, 20] + - [86, 20] + - [87, 20] + - [88, 20] + - [89, 20] + - [90, 20] + - [91, 20] + - [92, 20] + - [93, 20] + - [94, 20] + - [95, 20] + - [96, 20] + - [97, 20] + - [98, 20] + - [99, 20] + - [100, 20] + - [101, 20] + - [102, 20] + - [103, 20] + - [104, 20] + - [105, 20] + - [106, 20] + - [107, 20] + - [108, 20] + - [109, 20] + - [110, 20] + - [111, 20] + - [112, 20] + - [113, 20] + - [114, 20] + - [115, 20] + - [116, 20] + - [117, 20] + - [118, 20] + - [119, 20] + - [120, 20] + - [121, 20] + - [122, 20] + - [123, 20] + - [124, 20] + - [125, 20] + - [126, 20] + - [127, 20] + - [0, 21] + - [1, 21] + - [2, 21] + - [3, 21] + - [4, 21] + - [5, 21] + - [6, 21] + - [7, 21] + - [8, 21] + - [9, 21] + - [10, 21] + - [11, 21] + - [12, 21] + - [13, 21] + - [14, 21] + - [15, 21] + - [16, 21] + - [17, 21] + - [18, 21] + - [19, 21] + - [20, 21] + - [21, 21] + - [22, 21] + - [23, 21] + - [24, 21] + - [25, 21] + - [26, 21] + - [27, 21] + - [28, 21] + - [29, 21] + - [30, 21] + - [31, 21] + - [32, 21] + - [33, 21] + - [34, 21] + - [35, 21] + - [36, 21] + - [37, 21] + - [38, 21] + - [39, 21] + - [40, 21] + - [41, 21] + - [42, 21] + - [43, 21] + - [44, 21] + - [45, 21] + - [46, 21] + - [47, 21] + - [48, 21] + - [49, 21] + - [50, 21] + - [51, 21] + - [52, 21] + - [53, 21] + - [54, 21] + - [55, 21] + - [56, 21] + - [57, 21] + - [58, 21] + - [59, 21] + - [64, 21] + - [65, 21] + - [66, 21] + - [67, 21] + - [68, 21] + - [69, 21] + - [70, 21] + - [71, 21] + - [72, 21] + - [73, 21] + - [74, 21] + - [75, 21] + - [76, 21] + - [77, 21] + - [78, 21] + - [79, 21] + - [80, 21] + - [81, 21] + - [82, 21] + - [83, 21] + - [84, 21] + - [85, 21] + - [86, 21] + - [87, 21] + - [88, 21] + - [89, 21] + - [90, 21] + - [91, 21] + - [92, 21] + - [93, 21] + - [94, 21] + - [95, 21] + - [96, 21] + - [97, 21] + - [98, 21] + - [99, 21] + - [100, 21] + - [101, 21] + - [102, 21] + - [103, 21] + - [104, 21] + - [105, 21] + - [106, 21] + - [107, 21] + - [108, 21] + - [109, 21] + - [110, 21] + - [111, 21] + - [112, 21] + - [113, 21] + - [114, 21] + - [115, 21] + - [116, 21] + - [117, 21] + - [118, 21] + - [119, 21] + - [120, 21] + - [121, 21] + - [122, 21] + - [123, 21] + - [124, 21] + - [125, 21] + - [126, 21] + - [127, 21] + win_condition: + all_of: + - units_killed_gte: 4 + - has_building: fact + - within_ticks: 2700 + fail_condition: + any_of: + - after_ticks: 2701 + - not: + has_building: fact + max_turns: 35 + medium: + description: Two bridges (west-of-centre and east-of-centre) cross the water. Each bridge has a refinery (proc) directly + behind it that the attackers will destroy if unopposed. Six riflemen attack through EACH bridge. Defenders are on HoldFire + (stance 0) at the centre; you must split + flip stance + intercept at both bridges. Win = kill ≥8 attackers AND keep + BOTH procs alive, before turn 35. + starting_cash: 1000 + overrides: + actors: + - *id001 + - *id002 + - type: proc + owner: agent + position: + - 40 + - 26 + - type: proc + owner: agent + position: + - 82 + - 26 + - type: e1 + owner: agent + position: + - 59 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 62 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 63 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 64 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 65 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 59 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 62 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 63 + - 31 + stance: 0 + - type: e1 + owner: enemy + position: + - 38 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 39 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 40 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 41 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 42 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 43 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 80 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 81 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 82 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 83 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 84 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 85 + - 4 + stance: 3 + - type: fact + owner: enemy + position: + - 120 + - 4 + water_cells: + - [0, 18] + - [1, 18] + - [2, 18] + - [3, 18] + - [4, 18] + - [5, 18] + - [6, 18] + - [7, 18] + - [8, 18] + - [9, 18] + - [10, 18] + - [11, 18] + - [12, 18] + - [13, 18] + - [14, 18] + - [15, 18] + - [16, 18] + - [17, 18] + - [18, 18] + - [19, 18] + - [20, 18] + - [21, 18] + - [22, 18] + - [23, 18] + - [24, 18] + - [25, 18] + - [26, 18] + - [27, 18] + - [28, 18] + - [29, 18] + - [30, 18] + - [31, 18] + - [32, 18] + - [33, 18] + - [34, 18] + - [35, 18] + - [36, 18] + - [37, 18] + - [38, 18] + - [39, 18] + - [44, 18] + - [45, 18] + - [46, 18] + - [47, 18] + - [48, 18] + - [49, 18] + - [50, 18] + - [51, 18] + - [52, 18] + - [53, 18] + - [54, 18] + - [55, 18] + - [56, 18] + - [57, 18] + - [58, 18] + - [59, 18] + - [60, 18] + - [61, 18] + - [62, 18] + - [63, 18] + - [64, 18] + - [65, 18] + - [66, 18] + - [67, 18] + - [68, 18] + - [69, 18] + - [70, 18] + - [71, 18] + - [72, 18] + - [73, 18] + - [74, 18] + - [75, 18] + - [76, 18] + - [77, 18] + - [78, 18] + - [79, 18] + - [80, 18] + - [81, 18] + - [86, 18] + - [87, 18] + - [88, 18] + - [89, 18] + - [90, 18] + - [91, 18] + - [92, 18] + - [93, 18] + - [94, 18] + - [95, 18] + - [96, 18] + - [97, 18] + - [98, 18] + - [99, 18] + - [100, 18] + - [101, 18] + - [102, 18] + - [103, 18] + - [104, 18] + - [105, 18] + - [106, 18] + - [107, 18] + - [108, 18] + - [109, 18] + - [110, 18] + - [111, 18] + - [112, 18] + - [113, 18] + - [114, 18] + - [115, 18] + - [116, 18] + - [117, 18] + - [118, 18] + - [119, 18] + - [120, 18] + - [121, 18] + - [122, 18] + - [123, 18] + - [124, 18] + - [125, 18] + - [126, 18] + - [127, 18] + - [0, 19] + - [1, 19] + - [2, 19] + - [3, 19] + - [4, 19] + - [5, 19] + - [6, 19] + - [7, 19] + - [8, 19] + - [9, 19] + - [10, 19] + - [11, 19] + - [12, 19] + - [13, 19] + - [14, 19] + - [15, 19] + - [16, 19] + - [17, 19] + - [18, 19] + - [19, 19] + - [20, 19] + - [21, 19] + - [22, 19] + - [23, 19] + - [24, 19] + - [25, 19] + - [26, 19] + - [27, 19] + - [28, 19] + - [29, 19] + - [30, 19] + - [31, 19] + - [32, 19] + - [33, 19] + - [34, 19] + - [35, 19] + - [36, 19] + - [37, 19] + - [38, 19] + - [39, 19] + - [44, 19] + - [45, 19] + - [46, 19] + - [47, 19] + - [48, 19] + - [49, 19] + - [50, 19] + - [51, 19] + - [52, 19] + - [53, 19] + - [54, 19] + - [55, 19] + - [56, 19] + - [57, 19] + - [58, 19] + - [59, 19] + - [60, 19] + - [61, 19] + - [62, 19] + - [63, 19] + - [64, 19] + - [65, 19] + - [66, 19] + - [67, 19] + - [68, 19] + - [69, 19] + - [70, 19] + - [71, 19] + - [72, 19] + - [73, 19] + - [74, 19] + - [75, 19] + - [76, 19] + - [77, 19] + - [78, 19] + - [79, 19] + - [80, 19] + - [81, 19] + - [86, 19] + - [87, 19] + - [88, 19] + - [89, 19] + - [90, 19] + - [91, 19] + - [92, 19] + - [93, 19] + - [94, 19] + - [95, 19] + - [96, 19] + - [97, 19] + - [98, 19] + - [99, 19] + - [100, 19] + - [101, 19] + - [102, 19] + - [103, 19] + - [104, 19] + - [105, 19] + - [106, 19] + - [107, 19] + - [108, 19] + - [109, 19] + - [110, 19] + - [111, 19] + - [112, 19] + - [113, 19] + - [114, 19] + - [115, 19] + - [116, 19] + - [117, 19] + - [118, 19] + - [119, 19] + - [120, 19] + - [121, 19] + - [122, 19] + - [123, 19] + - [124, 19] + - [125, 19] + - [126, 19] + - [127, 19] + - [0, 20] + - [1, 20] + - [2, 20] + - [3, 20] + - [4, 20] + - [5, 20] + - [6, 20] + - [7, 20] + - [8, 20] + - [9, 20] + - [10, 20] + - [11, 20] + - [12, 20] + - [13, 20] + - [14, 20] + - [15, 20] + - [16, 20] + - [17, 20] + - [18, 20] + - [19, 20] + - [20, 20] + - [21, 20] + - [22, 20] + - [23, 20] + - [24, 20] + - [25, 20] + - [26, 20] + - [27, 20] + - [28, 20] + - [29, 20] + - [30, 20] + - [31, 20] + - [32, 20] + - [33, 20] + - [34, 20] + - [35, 20] + - [36, 20] + - [37, 20] + - [38, 20] + - [39, 20] + - [44, 20] + - [45, 20] + - [46, 20] + - [47, 20] + - [48, 20] + - [49, 20] + - [50, 20] + - [51, 20] + - [52, 20] + - [53, 20] + - [54, 20] + - [55, 20] + - [56, 20] + - [57, 20] + - [58, 20] + - [59, 20] + - [60, 20] + - [61, 20] + - [62, 20] + - [63, 20] + - [64, 20] + - [65, 20] + - [66, 20] + - [67, 20] + - [68, 20] + - [69, 20] + - [70, 20] + - [71, 20] + - [72, 20] + - [73, 20] + - [74, 20] + - [75, 20] + - [76, 20] + - [77, 20] + - [78, 20] + - [79, 20] + - [80, 20] + - [81, 20] + - [86, 20] + - [87, 20] + - [88, 20] + - [89, 20] + - [90, 20] + - [91, 20] + - [92, 20] + - [93, 20] + - [94, 20] + - [95, 20] + - [96, 20] + - [97, 20] + - [98, 20] + - [99, 20] + - [100, 20] + - [101, 20] + - [102, 20] + - [103, 20] + - [104, 20] + - [105, 20] + - [106, 20] + - [107, 20] + - [108, 20] + - [109, 20] + - [110, 20] + - [111, 20] + - [112, 20] + - [113, 20] + - [114, 20] + - [115, 20] + - [116, 20] + - [117, 20] + - [118, 20] + - [119, 20] + - [120, 20] + - [121, 20] + - [122, 20] + - [123, 20] + - [124, 20] + - [125, 20] + - [126, 20] + - [127, 20] + - [0, 21] + - [1, 21] + - [2, 21] + - [3, 21] + - [4, 21] + - [5, 21] + - [6, 21] + - [7, 21] + - [8, 21] + - [9, 21] + - [10, 21] + - [11, 21] + - [12, 21] + - [13, 21] + - [14, 21] + - [15, 21] + - [16, 21] + - [17, 21] + - [18, 21] + - [19, 21] + - [20, 21] + - [21, 21] + - [22, 21] + - [23, 21] + - [24, 21] + - [25, 21] + - [26, 21] + - [27, 21] + - [28, 21] + - [29, 21] + - [30, 21] + - [31, 21] + - [32, 21] + - [33, 21] + - [34, 21] + - [35, 21] + - [36, 21] + - [37, 21] + - [38, 21] + - [39, 21] + - [44, 21] + - [45, 21] + - [46, 21] + - [47, 21] + - [48, 21] + - [49, 21] + - [50, 21] + - [51, 21] + - [52, 21] + - [53, 21] + - [54, 21] + - [55, 21] + - [56, 21] + - [57, 21] + - [58, 21] + - [59, 21] + - [60, 21] + - [61, 21] + - [62, 21] + - [63, 21] + - [64, 21] + - [65, 21] + - [66, 21] + - [67, 21] + - [68, 21] + - [69, 21] + - [70, 21] + - [71, 21] + - [72, 21] + - [73, 21] + - [74, 21] + - [75, 21] + - [76, 21] + - [77, 21] + - [78, 21] + - [79, 21] + - [80, 21] + - [81, 21] + - [86, 21] + - [87, 21] + - [88, 21] + - [89, 21] + - [90, 21] + - [91, 21] + - [92, 21] + - [93, 21] + - [94, 21] + - [95, 21] + - [96, 21] + - [97, 21] + - [98, 21] + - [99, 21] + - [100, 21] + - [101, 21] + - [102, 21] + - [103, 21] + - [104, 21] + - [105, 21] + - [106, 21] + - [107, 21] + - [108, 21] + - [109, 21] + - [110, 21] + - [111, 21] + - [112, 21] + - [113, 21] + - [114, 21] + - [115, 21] + - [116, 21] + - [117, 21] + - [118, 21] + - [119, 21] + - [120, 21] + - [121, 21] + - [122, 21] + - [123, 21] + - [124, 21] + - [125, 21] + - [126, 21] + - [127, 21] + win_condition: + all_of: + - units_killed_gte: 8 + - has_building: fact + - within_ticks: 3150 + - building_count_gte: + type: proc + n: 2 + fail_condition: + any_of: + - after_ticks: 3151 + - not: + has_building: fact + - not: + building_count_gte: + type: proc + n: 2 + max_turns: 40 + hard: + description: Three bridges (west, centre, east) cross the water. Each bridge has a refinery (proc) directly behind it + that the attackers will destroy if unopposed. Five riflemen attack through EACH bridge. Defenders are on HoldFire (stance + 0) at the centre. You must split into thirds + flip stance + intercept at all three bridges. Win = kill ≥10 attackers + AND keep ALL THREE procs alive, before turn 40. + starting_cash: 1000 + overrides: + actors: + - *id001 + - *id002 + - type: proc + owner: agent + position: + - 28 + - 26 + - type: proc + owner: agent + position: + - 60 + - 26 + - type: proc + owner: agent + position: + - 92 + - 26 + - type: e1 + owner: agent + position: + - 58 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 59 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 62 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 63 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 64 + - 30 + stance: 0 + - type: e1 + owner: agent + position: + - 58 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 59 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 62 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 63 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 64 + - 31 + stance: 0 + - type: e1 + owner: agent + position: + - 58 + - 32 + stance: 0 + - type: e1 + owner: agent + position: + - 59 + - 32 + stance: 0 + - type: e1 + owner: agent + position: + - 60 + - 32 + stance: 0 + - type: e1 + owner: agent + position: + - 61 + - 32 + stance: 0 + - type: e1 + owner: enemy + position: + - 26 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 27 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 28 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 29 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 30 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 58 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 59 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 60 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 61 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 62 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 90 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 91 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 92 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 93 + - 4 + stance: 3 + - type: e1 + owner: enemy + position: + - 94 + - 4 + stance: 3 + - type: fact + owner: enemy + position: + - 120 + - 4 + water_cells: + - [0, 18] + - [1, 18] + - [2, 18] + - [3, 18] + - [4, 18] + - [5, 18] + - [6, 18] + - [7, 18] + - [8, 18] + - [9, 18] + - [10, 18] + - [11, 18] + - [12, 18] + - [13, 18] + - [14, 18] + - [15, 18] + - [16, 18] + - [17, 18] + - [18, 18] + - [19, 18] + - [20, 18] + - [21, 18] + - [22, 18] + - [23, 18] + - [24, 18] + - [25, 18] + - [26, 18] + - [27, 18] + - [32, 18] + - [33, 18] + - [34, 18] + - [35, 18] + - [36, 18] + - [37, 18] + - [38, 18] + - [39, 18] + - [40, 18] + - [41, 18] + - [42, 18] + - [43, 18] + - [44, 18] + - [45, 18] + - [46, 18] + - [47, 18] + - [48, 18] + - [49, 18] + - [50, 18] + - [51, 18] + - [52, 18] + - [53, 18] + - [54, 18] + - [55, 18] + - [56, 18] + - [57, 18] + - [58, 18] + - [59, 18] + - [64, 18] + - [65, 18] + - [66, 18] + - [67, 18] + - [68, 18] + - [69, 18] + - [70, 18] + - [71, 18] + - [72, 18] + - [73, 18] + - [74, 18] + - [75, 18] + - [76, 18] + - [77, 18] + - [78, 18] + - [79, 18] + - [80, 18] + - [81, 18] + - [82, 18] + - [83, 18] + - [84, 18] + - [85, 18] + - [86, 18] + - [87, 18] + - [88, 18] + - [89, 18] + - [90, 18] + - [91, 18] + - [96, 18] + - [97, 18] + - [98, 18] + - [99, 18] + - [100, 18] + - [101, 18] + - [102, 18] + - [103, 18] + - [104, 18] + - [105, 18] + - [106, 18] + - [107, 18] + - [108, 18] + - [109, 18] + - [110, 18] + - [111, 18] + - [112, 18] + - [113, 18] + - [114, 18] + - [115, 18] + - [116, 18] + - [117, 18] + - [118, 18] + - [119, 18] + - [120, 18] + - [121, 18] + - [122, 18] + - [123, 18] + - [124, 18] + - [125, 18] + - [126, 18] + - [127, 18] + - [0, 19] + - [1, 19] + - [2, 19] + - [3, 19] + - [4, 19] + - [5, 19] + - [6, 19] + - [7, 19] + - [8, 19] + - [9, 19] + - [10, 19] + - [11, 19] + - [12, 19] + - [13, 19] + - [14, 19] + - [15, 19] + - [16, 19] + - [17, 19] + - [18, 19] + - [19, 19] + - [20, 19] + - [21, 19] + - [22, 19] + - [23, 19] + - [24, 19] + - [25, 19] + - [26, 19] + - [27, 19] + - [32, 19] + - [33, 19] + - [34, 19] + - [35, 19] + - [36, 19] + - [37, 19] + - [38, 19] + - [39, 19] + - [40, 19] + - [41, 19] + - [42, 19] + - [43, 19] + - [44, 19] + - [45, 19] + - [46, 19] + - [47, 19] + - [48, 19] + - [49, 19] + - [50, 19] + - [51, 19] + - [52, 19] + - [53, 19] + - [54, 19] + - [55, 19] + - [56, 19] + - [57, 19] + - [58, 19] + - [59, 19] + - [64, 19] + - [65, 19] + - [66, 19] + - [67, 19] + - [68, 19] + - [69, 19] + - [70, 19] + - [71, 19] + - [72, 19] + - [73, 19] + - [74, 19] + - [75, 19] + - [76, 19] + - [77, 19] + - [78, 19] + - [79, 19] + - [80, 19] + - [81, 19] + - [82, 19] + - [83, 19] + - [84, 19] + - [85, 19] + - [86, 19] + - [87, 19] + - [88, 19] + - [89, 19] + - [90, 19] + - [91, 19] + - [96, 19] + - [97, 19] + - [98, 19] + - [99, 19] + - [100, 19] + - [101, 19] + - [102, 19] + - [103, 19] + - [104, 19] + - [105, 19] + - [106, 19] + - [107, 19] + - [108, 19] + - [109, 19] + - [110, 19] + - [111, 19] + - [112, 19] + - [113, 19] + - [114, 19] + - [115, 19] + - [116, 19] + - [117, 19] + - [118, 19] + - [119, 19] + - [120, 19] + - [121, 19] + - [122, 19] + - [123, 19] + - [124, 19] + - [125, 19] + - [126, 19] + - [127, 19] + - [0, 20] + - [1, 20] + - [2, 20] + - [3, 20] + - [4, 20] + - [5, 20] + - [6, 20] + - [7, 20] + - [8, 20] + - [9, 20] + - [10, 20] + - [11, 20] + - [12, 20] + - [13, 20] + - [14, 20] + - [15, 20] + - [16, 20] + - [17, 20] + - [18, 20] + - [19, 20] + - [20, 20] + - [21, 20] + - [22, 20] + - [23, 20] + - [24, 20] + - [25, 20] + - [26, 20] + - [27, 20] + - [32, 20] + - [33, 20] + - [34, 20] + - [35, 20] + - [36, 20] + - [37, 20] + - [38, 20] + - [39, 20] + - [40, 20] + - [41, 20] + - [42, 20] + - [43, 20] + - [44, 20] + - [45, 20] + - [46, 20] + - [47, 20] + - [48, 20] + - [49, 20] + - [50, 20] + - [51, 20] + - [52, 20] + - [53, 20] + - [54, 20] + - [55, 20] + - [56, 20] + - [57, 20] + - [58, 20] + - [59, 20] + - [64, 20] + - [65, 20] + - [66, 20] + - [67, 20] + - [68, 20] + - [69, 20] + - [70, 20] + - [71, 20] + - [72, 20] + - [73, 20] + - [74, 20] + - [75, 20] + - [76, 20] + - [77, 20] + - [78, 20] + - [79, 20] + - [80, 20] + - [81, 20] + - [82, 20] + - [83, 20] + - [84, 20] + - [85, 20] + - [86, 20] + - [87, 20] + - [88, 20] + - [89, 20] + - [90, 20] + - [91, 20] + - [96, 20] + - [97, 20] + - [98, 20] + - [99, 20] + - [100, 20] + - [101, 20] + - [102, 20] + - [103, 20] + - [104, 20] + - [105, 20] + - [106, 20] + - [107, 20] + - [108, 20] + - [109, 20] + - [110, 20] + - [111, 20] + - [112, 20] + - [113, 20] + - [114, 20] + - [115, 20] + - [116, 20] + - [117, 20] + - [118, 20] + - [119, 20] + - [120, 20] + - [121, 20] + - [122, 20] + - [123, 20] + - [124, 20] + - [125, 20] + - [126, 20] + - [127, 20] + - [0, 21] + - [1, 21] + - [2, 21] + - [3, 21] + - [4, 21] + - [5, 21] + - [6, 21] + - [7, 21] + - [8, 21] + - [9, 21] + - [10, 21] + - [11, 21] + - [12, 21] + - [13, 21] + - [14, 21] + - [15, 21] + - [16, 21] + - [17, 21] + - [18, 21] + - [19, 21] + - [20, 21] + - [21, 21] + - [22, 21] + - [23, 21] + - [24, 21] + - [25, 21] + - [26, 21] + - [27, 21] + - [32, 21] + - [33, 21] + - [34, 21] + - [35, 21] + - [36, 21] + - [37, 21] + - [38, 21] + - [39, 21] + - [40, 21] + - [41, 21] + - [42, 21] + - [43, 21] + - [44, 21] + - [45, 21] + - [46, 21] + - [47, 21] + - [48, 21] + - [49, 21] + - [50, 21] + - [51, 21] + - [52, 21] + - [53, 21] + - [54, 21] + - [55, 21] + - [56, 21] + - [57, 21] + - [58, 21] + - [59, 21] + - [64, 21] + - [65, 21] + - [66, 21] + - [67, 21] + - [68, 21] + - [69, 21] + - [70, 21] + - [71, 21] + - [72, 21] + - [73, 21] + - [74, 21] + - [75, 21] + - [76, 21] + - [77, 21] + - [78, 21] + - [79, 21] + - [80, 21] + - [81, 21] + - [82, 21] + - [83, 21] + - [84, 21] + - [85, 21] + - [86, 21] + - [87, 21] + - [88, 21] + - [89, 21] + - [90, 21] + - [91, 21] + - [96, 21] + - [97, 21] + - [98, 21] + - [99, 21] + - [100, 21] + - [101, 21] + - [102, 21] + - [103, 21] + - [104, 21] + - [105, 21] + - [106, 21] + - [107, 21] + - [108, 21] + - [109, 21] + - [110, 21] + - [111, 21] + - [112, 21] + - [113, 21] + - [114, 21] + - [115, 21] + - [116, 21] + - [117, 21] + - [118, 21] + - [119, 21] + - [120, 21] + - [121, 21] + - [122, 21] + - [123, 21] + - [124, 21] + - [125, 21] + - [126, 21] + - [127, 21] + win_condition: + all_of: + - units_killed_gte: 10 + - has_building: fact + - within_ticks: 3600 + - building_count_gte: + type: proc + n: 3 + fail_condition: + any_of: + - after_ticks: 3601 + - not: + has_building: fact + - not: + building_count_gte: + type: proc + n: 3 + max_turns: 45 diff --git a/openra_bench/scenarios/packs/econ-contested-expansion.yaml b/openra_bench/scenarios/packs/econ-contested-expansion.yaml new file mode 100644 index 0000000000000000000000000000000000000000..f1f4166a93b179b942cbc5f23e8370daaef09b43 --- /dev/null +++ b/openra_bench/scenarios/packs/econ-contested-expansion.yaml @@ -0,0 +1,383 @@ +# ECON-CONTESTED-EXPANSION — second refinery for the second harvester. +# +# REASONING capability — mid-game expansion under threat. The agent +# owns a working mini-base (fact + powr + proc + harv) plus ONE small +# home ore patch whose single-harvester throughput is below the cash +# bar. A SECOND, RICHER ore patch sits midway between the agent and +# the enemy, patrolled by hostile infantry; the agent observes both +# patches on the `ore_cells` channel. The intended play is to BUILD A +# SECOND `proc` and `place_building` it — anywhere, but the contested +# patch is the obviously-tempting site — because the engine ships the +# new refinery with a FREE second harvester (`order_place_building` +# calls `spawn_unit("harv")` synchronously at line 1779 of world.rs). +# The two-harv throughput on the home patch is what crosses the bar; +# the contested patch is the visible "richer alternative" that lures +# a naive `send the home harv there` policy into the patrol's fire. +# +# THE NO-CHEAT BAR +# ---------------- +# Every lazy / brute / wrong-shape policy must LOSE on every level +# and every hard seed (1-4); the intended capability play must WIN. +# +# • stall (only observe): single home harv cycles the home patch +# and accumulates cash. The home-patch throughput ceiling + +# starting cash is 5630cr by tick 4000 (verified by scripted- +# policy probe, seed 1, 45 turns). Bars are 5800 / 6200 / 6500; +# stall LOSES on the after_ticks fail clause on every tier. +# +# • only-home (observe + queue silos at home but never build a 2nd +# proc): same income ceiling as stall — silos don't add throughput. +# LOSES. +# +# • send-harv-undefended (`harvest(harv, 60, 20)` — send the home +# harv to the contested patch without a refinery there and +# without an escort): the harv has to traverse ~45 cells one way +# and the engine's `find_refinery` returns the (lowest-id) home +# proc as the deposit point, so the round-trip is ~90 cells per +# bale. Effective throughput is LOWER than the original home +# cycle, AND the patrol e1 platoon engages the harv on contact. +# Stall-equivalent LOSS in best case, harv-death LOSS otherwise. +# +# • intended (`build('proc')` + `place_building('proc', x, y)`): +# the engine spawns a free 2nd `harv` adjacent to the OLD proc +# (engine quirk #1 below — the spawn site is the lowest-id +# production building, not the new building); the 2nd harv pairs +# with the original on the home patch, doubling throughput; the +# bar latches at cash >= 6500 by tick ~3600. WIN. +# +# ENGINE FACTS USED (verified against the installed openra_train wheel) +# --------------------------------------------------------------------- +# 1. `place_building('proc', x, y)` AUTO-SPAWNS a free `harv` via +# `World::spawn_unit("harv", owner)` at world.rs:1779. The spawn +# site is `find_spawn_location` which sorts production buildings +# by `(!is_primary, id)` — so without an explicit `set_primary` +# the NEW proc is NOT preferred; the auto-spawn harv lands at the +# OLD proc. (This is an engine gap vs the SC2-style "free +# worker at the new expansion" idiom — see ENGINE GAPS below. +# The pack's no-cheat bar holds anyway because the doubled-harv +# home throughput is enough to clear the cash bar while the +# single-harv stall ceiling is not.) +# 2. `find_refinery` returns the lowest-id `proc` (world.rs:5006); +# both harvs deposit to the OLD proc regardless of the new +# proc's location. So neither harv exploits the contested patch +# in the intended path — the doubled-throughput cycle is on the +# HOME patch. The contested patch is a visible decoy in the +# ore_cells channel; sending the lone harv there is a LOSS. +# 3. `proc` requires `powr` AND `fact` (gamerules.rs). Both are +# pre-placed for the agent at every tier. +# 4. `proc` footprint is 3×4. The contested patch is radius 4 +# (49 cells) so a footprint-overlapping placement still leaves +# non-blockaded ore — but as noted in (1)/(2) above, the +# placement coordinate doesn't change throughput, only the +# building count predicate. +# 5. Per-player cash plumbing: bench `agent: {cash: 2000}` is honored +# (per `tests/test_per_player_starting_cash.py`); without this +# line the bench's `PlayerSetup` defaults `agent.cash: 0`, the +# proc build queue stalls at 0 progress, and every policy LOSSes +# trivially — masking the real capability. Top-level +# `starting_cash: 2000` is the back-compat fallback. +# 6. `cash_gte: N` reads `economy.cash` (post-deposit drain). Bars +# are tuned ABOVE the single-harv home-patch ceiling (~5630 +# by tick 4000) and BELOW the two-harv ceiling (~6250-6600 by +# tick ~3500-3800). +# 7. Tick alignment (CLAUDE.md): engine advances ~90 ticks per +# decision turn, so `within_ticks ≤ 93 + 90·(max_turns − 1)`. +# `max_turns: 45` ⇒ ceiling 93 + 90·44 = 4053; `within_ticks +# 4000`, `after_ticks 4001` both bite as a real LOSS (no draw). +# 8. Persistent unarmed enemy `fact` at the far east edge prevents +# the engine's `ConquestVictoryConditions` from auto-`done`-ing +# on enemy-elimination before the win/fail evaluates (CLAUDE.md +# auto-done footgun). On the hard tier the sentinel is +# duplicated across both agent spawn_point groups. +# 9. `bot_type: patrol` (openra-sim/src/scripted_bot.rs :: +# ScriptedBehavior::Patrol) makes each enemy unit oscillate +# around its own spawn cell and engage intruders in weapon range. +# The patrol e1 platoon is positioned NEAR the contested patch +# so the send-harv-undefended path leads the harv directly into +# their fire envelope. +# 10. Hard tier ≥2-spawn-groups requirement (CLAUDE.md §7): two +# agent `spawn_point` groups round-robined by seed — NORTH base +# at y=14 (spawn_point 0) vs SOUTH base at y=26 (spawn_point 1). +# The home patch, contested patch, patrol platoon, and +# persistent enemy sentinel are duplicated across both groups +# (ore_patches don't honour spawn_point, and the persistent +# sentinel mirrors the agent's latitude per spawn group). A +# memorised y=20 opening fails on both hard seed groups. +# +# ENGINE GAPS (for follow-up; do not block this pack) +# --------------------------------------------------- +# G1. `place_building('proc')` auto-spawn site. Per task design the +# free harv should spawn "adjacent to the new proc" (the SC2 +# free-worker idiom). The engine instead spawns it at the +# LOWEST-id proc via `find_spawn_location`'s `(!is_primary, id)` +# sort. A scenario can not work around this from YAML — the +# auto-spawn happens synchronously in `order_place_building` +# before any `set_primary` order can run. Fix: `spawn_unit` +# should accept an optional preferred-location argument and +# `order_place_building` should pass the new building's +# footprint. Until fixed, the pack's intended capability is +# "build a 2nd proc to get a 2nd harv at the OLD base" — the +# contested-expansion framing is narrative. +# G2. `find_refinery` always returns the lowest-id proc, so harvs +# cannot be split across multiple refineries by position. Fix: +# nearest-refinery selection, or a per-harv refinery binding. +# +# Validate (no model / no network): +# python3 -m openra_bench.scenarios.validate \ +# openra_bench/scenarios/packs/econ-contested-expansion.yaml + +meta: + id: econ-contested-expansion + title: 'Econ Contested Expansion — Second Refinery for the Second Harvester' + capability: reasoning + real_world_meaning: > + Mid-game expansion under threat. The agent owns a working + mini-base (Construction Yard, Power Plant, Refinery, one + harvester) and a small home ore patch whose single-harvester + steady-state income survives the clock but never clears the + cash bar. A second, richer ore patch sits midway between the + agent and an enemy patrol that already loiters at the site. + The capability under test is to recognise that single-harv home + throughput is below the bar, to commit the 1400-credit + refinery spend (which ships with a free second harvester via + the engine's place_building hook), and to NOT take the bait of + sending the lone home harvester unescorted into the patrolled + contested site. Stalling, fortifying at home, or sending the + home harvester to the contested patch without a refinery there + all LOSE. + robotics_analogue: > + Provision a second processor to unblock the asset queue. An + autonomous extraction operation is running an undersized loop + at its home deposit; a richer deposit a long traverse away is + patrolled by a hostile platform. The operator's lever is the + processor budget — a second processor commission ships with a + co-deployed extractor, doubling cycle-rate at the home site. + Detouring the existing extractor to the patrolled site without + a local processor stalls the cycle (long return trip) and + risks loss to the patrol. Stalling at the single-extractor + rate never clears the day's revenue target. + benchmark_anchor: + - "SC2 contested expansion (take a 2nd resource under harass)" + - "facility-siting under adversarial contention" + - "lmgame-Bench expand-vs-defend tradeoff" + - "RTS second-refinery idiom (free-worker on-build)" + author: openra-bench + +base_map: rush-hour-arena +# Top-level starting_cash is the back-compat fallback. Per-player +# `agent: {cash: 2000}` is what the engine actually honors (see +# `tests/test_per_player_starting_cash.py`); without it the agent +# starts at 0cr and every policy LOSSes trivially. +starting_cash: 2000 + +base: + # Per-player cash IS load-bearing — must set agent.cash explicitly, + # else the bench PlayerSetup default of 0cr applies and the proc + # build queue stalls. Enemy cash stays 0 (the patrol bot does not + # build / spend). + agent: {faction: allies, cash: 2000} + # Patrol bot oscillates each enemy unit around its own spawn cell + # and engages intruders in range — the threat at the contested + # patch that makes send-harv-undefended a real LOSS. + enemy: {faction: soviet, cash: 0, bot_type: patrol} + # Toolset: build / place_building are the load-bearing verbs (the + # 2nd proc); move_units / harvest / set_stance / attack_move / + # attack_unit / sell are auxiliary (re-routing, optional escort + # moves, sell-and-rebuild paths); observe / stop are stall / cancel. + tools: [observe, build, place_building, move_units, harvest, set_stance, attack_move, attack_unit, sell, stop] + spawn_mcvs: false + planning: true + termination: {max_ticks: 8000} + actors: [] # every level supplies its own actor list via overrides. + +levels: + # ── EASY ───────────────────────────────────────────────────────── + # Bare reasoning cell: home patch (radius 3, amount 3000) at + # (25, 20) + contested patch (radius 4, amount 12000) at (60, 20) + # patrolled by 2× e1 north/south of the patch. Bar cash >= 5800, + # within_ticks 4000. max_turns 45 ⇒ ceiling 93+90·44=4053 > 4001 ✓. + # Scripted-policy probe (seed 1): + # stall → cash=5630, LOSS (single-harv ceiling) + # send-harv-undefended → cash=5630, LOSS (harv detoured; same ceiling) + # intended (2nd proc) → cash=6600, WIN (doubled throughput) + easy: + description: > + You own a Construction Yard ('fact'), Power Plant ('powr'), Ore + Refinery ('proc'), and one Ore Truck ('harv') on the WEST side + of the map. Your home Ore Truck is auto-harvesting a SMALL home + ore patch at (25, 20). A second, RICHER ore patch is visible on + the ore_cells channel at the CONTESTED CENTRE (60, 20) — but + two enemy patrol infantry oscillate around it and will engage + any unprotected unit that enters their fire envelope. You have + $2000. Build a SECOND Ore Refinery ('proc', cost 1400) and + place it on the map — the engine ships the new refinery with a + FREE second Ore Truck, and the doubled harvester throughput is + what clears the cash bar. Win = cash >= 5800 before tick 4000. + Stalling, fortifying at home, and sending your home harvester + to the contested patch without a refinery there all LOSE. + starting_cash: 2000 + overrides: + actors: + # AGENT base — west side, y=20 lane. + - {type: fact, owner: agent, position: [10, 20]} + - {type: powr, owner: agent, position: [10, 18]} + - {type: proc, owner: agent, position: [12, 20]} + # Home harv adjacent to proc — auto-routes via + # world.rs::auto_route_idle_harvesters the moment the proc + # is owned. + - {type: harv, owner: agent, position: [15, 20]} + # Patrol — 2 e1 (rifle infantry) flanking the contested + # patch (north + south). bot_type:patrol oscillates them + # around their spawn and engages intruders within range; e1 + # is weak vs buildings (the cell is not a "destroy 2nd proc" + # test, it is a "do not send the harv unescorted" test). + - {type: e1, owner: enemy, position: [60, 14]} + - {type: e1, owner: enemy, position: [60, 26]} + # Persistent far-edge enemy sentinel — anti-DRAW per CLAUDE.md. + - {type: fact, owner: enemy, position: [120, 20]} + ore_patches: + # Home patch — small (radius 3, amount 3000). Single-harv + # steady-state throughput by tick 4000 + starting cash = + # 5630cr ceiling (verified). Below the 5800 bar ⇒ stall LOSS. + - {x: 25, y: 20, amount: 3000, radius: 3} + # Contested patch — rich (radius 4, amount 12000, 49 cells + # × 12/cell = 588 ore ≈ 5100cr if any harvester actually + # cycled it; in this engine no harv reaches it on the + # intended path because find_refinery returns the lowest-id + # proc — the contested patch is visible-on-map decoy that + # lures the lazy send-harv-undefended play into the patrol. + - {x: 60, y: 20, amount: 12000, radius: 4} + win_condition: + all_of: + - has_building: proc + - building_count_gte: {type: proc, n: 2} + - cash_gte: 5800 + - within_ticks: 4000 + fail_condition: + any_of: + - after_ticks: 4001 + - not: {has_building: fact} + max_turns: 45 + + # ── MEDIUM ─────────────────────────────────────────────────────── + # Same geometry as easy, tighter bar (cash >= 6200). The doubled- + # throughput intended path still clears (~6250 by tick ~3800); + # single-harv stall ceiling unchanged at 5630. + medium: + description: > + Same geometry as easy (small home patch at (25, 20) + rich + contested patch at (60, 20) patrolled by two enemy infantry). + Tighter bar: cash >= 6200 before tick 4000. Stall, only-home, + and send-harv-undefended all LOSE on the single-harvester home + throughput ceiling (~5630cr); the intended play (build + place + a 2nd Ore Refinery to receive the free 2nd Ore Truck) doubles + home-patch throughput to clear the bar. + starting_cash: 2000 + overrides: + actors: + - {type: fact, owner: agent, position: [10, 20]} + - {type: powr, owner: agent, position: [10, 18]} + - {type: proc, owner: agent, position: [12, 20]} + - {type: harv, owner: agent, position: [15, 20]} + - {type: e1, owner: enemy, position: [60, 14]} + - {type: e1, owner: enemy, position: [60, 26]} + - {type: fact, owner: enemy, position: [120, 20]} + ore_patches: + - {x: 25, y: 20, amount: 3000, radius: 3} + - {x: 60, y: 20, amount: 12000, radius: 4} + win_condition: + all_of: + - has_building: proc + - building_count_gte: {type: proc, n: 2} + - cash_gte: 6200 + - within_ticks: 4000 + fail_condition: + any_of: + - after_ticks: 4001 + - not: {has_building: fact} + max_turns: 45 + + # ── HARD ───────────────────────────────────────────────────────── + # Two distinct seed-driven spawn groups via `spawn_point` (CLAUDE.md + # ≥2 hard-tier requirement). The AGENT base latitude is round- + # robined NORTH (y=14, spawn_point 0) vs SOUTH (y=26, spawn_point + # 1) by seed. Each spawn group ships its OWN home patch, contested + # patch, patrol platoon, and persistent enemy sentinel on the + # matching latitude. + # + # `ore_patches:` is top-level and does NOT honour `spawn_point` + # (CLAUDE.md), so all four patches (home N + contested N + home S + # + contested S) materialise on every seed. Only the patch on the + # agent's active latitude is within the home harv's reach; the + # off-latitude patches are harmless flavour. + # + # `e1` enemy patrol units and the persistent sentinel are + # duplicated per spawn_point group — without per-group enemy + # actors the spawn_point filter rule (per-owner) would drop them + # on the inactive side. With spawn_point on both agent and enemy + # sides, only the matching latitude's enemies + sentinel place. + # + # Bar cash >= 6500. Intended (2nd proc, any placement) reaches + # ~6600 by tick ~3600 on every seed; stall caps at 5500-5630 + # depending on auto-route timing for the spawn latitude. A + # memorised "drop the 2nd proc at (58, 20)" opening still works + # (placement coordinate is not load-bearing for the win — what + # matters is that the build + place sequence completes — but the + # SPATIAL VARIATION in the home / contested / patrol positions is + # what makes "always send harv to (60, 20)" wrong: on the wrong + # latitude the harvest order points at a no-ore cell, and the + # patrol platoon is on a different y-band so an escort sent at + # (60, 20) attacks empty space). + hard: + description: > + Your west-side mini-base spawns in the NORTH (y=14) or SOUTH + (y=26) half of the map depending on the seed. You own a + Construction Yard, a Power Plant, an Ore Refinery, and one + Ore Truck auto-harvesting YOUR latitude's small home ore patch + (at (25, 14) for NORTH; (25, 26) for SOUTH). A richer ore + patch sits midway across the map on YOUR latitude at (60, 14) + or (60, 26), patrolled by two enemy infantry. Win = cash >= + 6500 before tick 4000. Build a SECOND Ore Refinery — the + engine ships it with a free second Ore Truck and the doubled + home-patch throughput clears the bar. Stalling, fortifying at + home, sending your home harvester unescorted toward the + contested patch, and memorising the wrong latitude (y=20) + all LOSE. + starting_cash: 2000 + overrides: + actors: + # spawn_point 0 — NORTH base (y=14). Agent base + enemy + # patrol + persistent sentinel on the north latitude. + - {type: fact, owner: agent, position: [10, 14], spawn_point: 0} + - {type: powr, owner: agent, position: [10, 12], spawn_point: 0} + - {type: proc, owner: agent, position: [12, 14], spawn_point: 0} + - {type: harv, owner: agent, position: [15, 14], spawn_point: 0} + - {type: e1, owner: enemy, position: [60, 11], spawn_point: 0} + - {type: e1, owner: enemy, position: [60, 17], spawn_point: 0} + - {type: fact, owner: enemy, position: [120, 14], spawn_point: 0} + # spawn_point 1 — SOUTH base (y=26). Symmetric. + - {type: fact, owner: agent, position: [10, 26], spawn_point: 1} + - {type: powr, owner: agent, position: [10, 28], spawn_point: 1} + - {type: proc, owner: agent, position: [12, 26], spawn_point: 1} + - {type: harv, owner: agent, position: [15, 26], spawn_point: 1} + - {type: e1, owner: enemy, position: [60, 23], spawn_point: 1} + - {type: e1, owner: enemy, position: [60, 29], spawn_point: 1} + - {type: fact, owner: enemy, position: [120, 26], spawn_point: 1} + ore_patches: + # Both latitudes' patches materialise every seed; only the + # one on the agent's active latitude is reachable. + - {x: 25, y: 14, amount: 3000, radius: 3} + - {x: 60, y: 14, amount: 12000, radius: 4} + - {x: 25, y: 26, amount: 3000, radius: 3} + - {x: 60, y: 26, amount: 12000, radius: 4} + win_condition: + all_of: + - has_building: proc + - building_count_gte: {type: proc, n: 2} + - cash_gte: 6500 + - within_ticks: 4000 + fail_condition: + any_of: + - after_ticks: 4001 + - not: {has_building: fact} + max_turns: 45 diff --git a/openra_bench/scenarios/packs/econ-harvester-defense-raid.yaml b/openra_bench/scenarios/packs/econ-harvester-defense-raid.yaml new file mode 100644 index 0000000000000000000000000000000000000000..8b14337bc555c17bfa71f20585cc991a47363674 --- /dev/null +++ b/openra_bench/scenarios/packs/econ-harvester-defense-raid.yaml @@ -0,0 +1,442 @@ +# econ-harvester-defense-raid — keep the harv ALIVE under live raider waves. +# +# ACTION capability — live-economy defense under harvester harass. +# Differs from `econ-protect-harvester-route` (which tests CONVOY +# ESCORT across a long commute) and `mid-economy-under-fire` (which +# rings the patch with auto-firing tanks at base-sight): THIS pack +# tests STANCE management on the escort. The escort is pre-placed +# at the proc on stance:0 (HoldFire) — even with PERFECT positioning, +# a "place defenders and observe" policy LOSES because stance:0 units +# never auto-engage, so the raider chews through the harv unopposed. +# The load-bearing verb is `set_stance(escort, 2 or 3)` + escort +# movement onto the harv's lane. +# +# DESIGN +# ------ +# Agent has fact + powr + proc + 1× harv + ONE NEAR ore patch (mine +# at x=25). The proc seeds the harv's auto-harvest loop (per +# world.rs::auto_route_idle_harvesters — a scenario-placed idle harv +# auto-routes once its owner owns a refinery), so the harv cycles +# proc(12,20) ↔ mine(25,20) WITHOUT an explicit harvest order. Income +# is steady AS LONG AS THE HARV LIVES. +# +# Threat: enemy `bot_type: raider` (openra-sim/src/scripted_bot.rs: +# ScriptedBehavior::Raider) — its tick() picks the agent's harv +# specifically via `harv_foes()` and issues an Attack on the nearest +# one every 12 ticks (the raider interval). New units added to the +# raider's roster (via scheduled_events::spawn_actors) are picked up +# on the next own_mobile() scan, so SCHEDULED raid waves all chase +# the harv. +# +# Wave schedule (scheduled_events: spawn_actors): +# easy: 1 wave — 1× 2tnk at tick 600 from (40,20) +# medium: 2 waves — 1× 2tnk at tick 600, 1× 2tnk at tick 1500 — both at (40,20) +# hard: 3 waves on 2 seed-driven axes — see hard tier comment +# +# Escort: 2× 2tnk at base, stance:0 (HoldFire). The escort SITS at +# the proc; even when a raider walks into its weapon range the +# stance:0 units do not fire (verified per CLAUDE.md stance fix: +# stance:0 never auto-engages, even when shot at, even when an enemy +# is sitting on top of them). The agent must EITHER: +# (a) `set_stance(escort, 3)` — Hunt: idle escort actively advances +# on visible enemies (raider walks into LOS as it approaches +# the harv's lane, escort engages); OR +# (b) `set_stance(escort, 2)` AND `attack_move(escort, raider lane)` +# — Defend stance auto-fires on whatever enters weapon range, +# so move the escort east along the harv's commute axis to +# INTERCEPT the raid wave; OR +# (c) `attack_unit(escort, raider_id)` — explicit order overrides +# stance:0, escort closes and kills the raider. +# Any of (a), (b), (c) clears the bar. The capability is "flip the +# escort from passive to active" — same verb as combat-tanya-vs-rush. +# +# THE NO-CHEAT BAR (every lazy / stall / wrong-shape policy LOSES on +# every level and every hard seed; the intended capability WINS): +# +# • stall (only observe): escort stays stance:0 at base, never +# engages. Raider wave 1 reaches the harv around tick ~750 +# (40→16 ≈ 24 cells at 2tnk speed) and kills it. unit_type_count_gte +# :{harv,1} fails → fail_condition fires → LOSS. +# +# • defensive-but-passive (place / re-issue HoldFire on escort, +# observe): identical outcome — the escort is structurally +# incapable of engaging. Harv dies → LOSS. +# +# • aggressive-but-no-escort (send escort east to the raider's +# spawn point at x=40, leaving harv alone on the lane between +# proc(12,20) and mine(25,20)): the escort marches AROUND or +# PAST the next raid wave, the wave reaches the undefended harv, +# harv dies → LOSS. (On easy with one wave the escort might +# catch the lone raider, but on medium/hard, the second/third +# wave finds the harv unprotected because the escort overshot.) +# +# • intended (set_stance(escort, 2 or 3) and/or attack_move escort +# onto the harv's lane ~x=20..25): raider walks into the escort's +# fire envelope on its approach, escort kills the raid wave +# before contact with the harv, harv survives, auto-route loop +# continues, cash bar clears → WIN. +# +# ENGINE FACTS (CLAUDE.md): +# - Tick budget ~90/turn. easy max_turns=35 ⇒ ceiling 93+90·34=3153; +# within_ticks:3000, after_ticks:3001 both bite. medium=45 ⇒ +# ceiling 4053; within_ticks:3900, after_ticks:3901. hard=50 ⇒ +# ceiling 4503; within_ticks:4300, after_ticks:4301. +# - `auto_route_idle_harvesters` auto-installs a Harvest activity +# on any owned idle harv whose owner owns a proc — no explicit +# harvest order needed. The harvest tool is provided so the +# agent CAN re-issue if needed, but is not strictly required. +# - `bot_type: raider` issues an Attack on the nearest agent harv +# every 12 ticks; new bot units added mid-episode (via +# scheduled_events::spawn_actors) join the bot's tick loop. +# - stance:0 (HoldFire) per CLAUDE.md never auto-engages — the +# "defensive but passive" failure mode is structurally enforced +# by the pre-placed stance. +# - stance:2/3 (Defend / AttackAnything) per CLAUDE.md: stance:2 +# auto-fires on in-range enemies but does not advance; stance:3 +# additionally advances on visible enemies. Either is a valid +# intended path; the load-bearing verb is the stance flip OR +# an explicit attack order (which overrides stance per the +# CLAUDE.md stance fix). +# - Persistent far enemy `fact` at (120,20) — anti-DRAW per +# CLAUDE.md (engine auto-`done`s on enemy-elim before win/fail +# evaluates; the inert far marker keeps the episode alive past +# the raider being destroyed). +# - `unit_type_count_gte: {harv, 1}` in fail_condition is the +# hard harv-survival predicate — the moment the harv dies the +# fail clause fires and the LOSS is immediate. +# - hard tier uses TWO seed-driven SCHEDULED_EVENTS spawn AXES +# (raid approaches from the north vs the south of the harv's +# lane). The `spawn_actors` event itself doesn't honour +# spawn_point, so per-seed axis variation is done by giving +# each seed group a distinct AGENT spawn_point — the active +# base latitude (y=14 vs y=26) determines which raid lane is +# "the harv's lane". This satisfies the ≥2-distinct-seed-driven- +# spawn-groups hard-tier requirement (CLAUDE.md §7). +# +# Validate (no model / no network): +# python3 -m openra_bench.scenarios.validate \ +# openra_bench/scenarios/packs/econ-harvester-defense-raid.yaml + +meta: + id: econ-harvester-defense-raid + title: 'Econ Harvester Defense — Keep the Harv Alive Under Raid Waves' + capability: action + real_world_meaning: > + A live extraction operation under recurring asset-targeted + harass. The agent owns a refinery, a single harvester running an + auto-harvest loop on a nearby ore patch, and a small escort + parked at the depot on hold-fire. The harvester's income is the + sole revenue stream, and a recurring raider drives at the + harvester specifically (not the base, not the escort). The + decision is to FLIP the escort from passive to active — set its + stance to defend / attack-anything, or order it onto the + harvester's lane to intercept — so the raider dies on the + escort's fire envelope before reaching the harvester. Stalling + or leaving the escort on hold-fire lets every wave through; the + harvester dies, income stops, the cash bar is unmet. + robotics_analogue: > + Mining-fleet protection under recurring asset-targeted attack — + an autonomous extraction site with an extractor robot, a + processing depot, and a static defense detail. If the defense + detail is in safe-mode (hold fire) the recurring attacker + destroys the extractor on the first cycle; the operator must + flip the detail to engage and (optionally) reposition it onto + the extractor's commute lane. + benchmark_anchor: + - "SC2 harvester harass defense (Lambo, Reaper drops on workers)" + - "convoy / extractor protection under recurring harass" + - "RTS stance management — flip escort from passive to active" + - "SOC active-vs-passive monitoring posture" + author: openra-bench + +base_map: rush-hour-arena +starting_cash: 0 # ALL income via harv survival; bar measures live throughput. + +base: + agent: {faction: allies, cash: 0} + # Enemy bot is `raider` — scripted_bot.rs ScriptedBehavior::Raider + # targets the agent's harv specifically via harv_foes(); new + # actors added via scheduled_events::spawn_actors are picked up + # by own_mobile() on the next tick, so every raid wave chases the + # harv. + enemy: {faction: soviet, cash: 0, bot_type: raider} + # Toolset: harvest is provided (the harv auto-routes from t=0 once + # the proc exists, but the agent can re-issue if needed); + # set_stance is the load-bearing verb; attack_unit / attack_move + # are valid alternates (explicit orders override stance:0). + tools: [observe, harvest, move_units, attack_unit, attack_move, set_stance, stop] + planning: true + # Interrupts left disabled — per econ-protect-harvester-route, the + # per-turn tick rate would collapse on enemy_unit_spotted under + # recurring raid waves and the cash bar wouldn't be reachable. + interrupts: {} + termination: {max_ticks: 6000} + actors: [] # every level supplies its own actor list via overrides. + +levels: + # ── EASY ───────────────────────────────────────────────────────── + # Bare skill: ONE raid wave (1× 2tnk) at tick 600. Harv cycles + # proc(12,20)↔mine(25,20) on auto-route. Escort: 2× 2tnk at base + # stance:0. Stall / passive: raider reaches harv ~tick 750, harv + # dies, LOSS. Intended: set_stance(escort, 2 or 3) and/or + # attack_move escort east to ~x=20, escort engages the raider on + # approach, harv survives, cash accumulates from auto-harvest. + # Bar: cash >= 800 (~2 refines × ~400cr) before tick 3000. + # max_turns 35 ⇒ ceiling 93+90·34=3153 > 3001 ✓. + easy: + description: > + You own a Construction Yard, Power Plant, Refinery, ONE Ore + Truck on auto-harvest to the near patch at (25,20), and TWO + medium tanks (2tnk) parked at your refinery on HOLD-FIRE + stance. At tick 600 an enemy raider tank spawns at (40,20) + and drives STRAIGHT at your harvester (the raider bot targets + harvesters specifically). Win when cash >= 800 AND your + harvester is still alive AND your refinery is intact, before + tick 3000. Standing the escort idle (HOLD-FIRE) lets the + raider walk through and kill the harvester. The intended play + is to FLIP the escort's stance to Defend (2) or AttackAnything + (3), OR explicitly attack-order it onto the raider's approach + lane. + starting_cash: 0 + overrides: + actors: + # Agent base — fact + powr seed the construction & power; + # proc seeds the harv auto-harvest loop. + - {type: fact, owner: agent, position: [10, 20]} + - {type: powr, owner: agent, position: [10, 18]} + - {type: proc, owner: agent, position: [12, 20]} + # Single harv — pre-placed adjacent to proc. With proc on the + # field, world.rs::auto_route_idle_harvesters installs a + # Harvest activity on the next tick; harv cycles proc<->mine. + - {type: harv, owner: agent, position: [14, 20]} + # Escort: 2× 2tnk at the proc on stance:0 (HoldFire). The + # capability under test is "flip the escort active" — stance:0 + # never auto-engages, so a "place defenders and observe" + # policy LOSES. + - {type: 2tnk, owner: agent, position: [13, 18], stance: 0} + - {type: 2tnk, owner: agent, position: [13, 22], stance: 0} + # Near ore patch — the harv's auto-route target. + - {type: mine, owner: neutral, position: [25, 20]} + # Persistent far enemy fact — anti-DRAW (engine auto-done + # mitigation; without this the run collapses to DRAW the + # instant the raid wave is destroyed, before the cash bar + # evaluates). + - {type: fact, owner: enemy, position: [120, 20]} + scheduled_events: + # Single raid wave at tick 600 — one 2tnk spawned east of + # the patch. Under bot_type:raider the new tank joins the + # raider's own_mobile() roster on the next tick and beelines + # toward the harv via harv_foes(). + - tick: 600 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + win_condition: + # Ordered `then:` enforces "harv must SURVIVE past the raid + # window and THEN keep refining to the bar" — clause 1 + # (after_ticks: 1700) latches only past wave 1's typical + # kill-window (raider spawns tick 600, reaches harv ~tick 900, + # kills undefended harv by ~tick 1500). A stall run loses its + # harv around tick ~1500 so by the time clause 1 latches the + # `unit_type_count_gte:{harv,1}` clause has already failed → + # fail_condition fires first. The intended escort engagement + # kills the raider, harv keeps cycling, cash crosses 1200 + # well after tick 1700, all_of latches → WIN. + all_of: + - then: + id: econ-harvester-defense-raid-easy + clauses: + - {after_ticks: 1700} + - {cash_gte: 1200} + - {unit_type_count_gte: {type: harv, n: 1}} + - {has_building: proc} + - {within_ticks: 3000} + fail_condition: + any_of: + - {after_ticks: 3001} + - {not: {unit_type_count_gte: {type: harv, n: 1}}} + - {not: {has_building: proc}} + max_turns: 35 + + # ── MEDIUM ─────────────────────────────────────────────────────── + # +1 controlled variable: TWO raid waves (tick 600, tick 1500) and + # a tighter bar (cash >= 1600 ≈ 4 refines). A one-time "kill the + # first raider" play has to ALSO survive the second wave — the + # escort must be RETURNED to a covering position (or kept on + # AttackAnything stance:3) so the late wave dies on approach too. + # max_turns 45 ⇒ ceiling 4053 > 3901 ✓. + medium: + description: > + Same setup as easy — fact + powr + proc + 1× harv on auto- + harvest to the near patch at (25,20), 2× 2tnk escort at the + refinery on HOLD-FIRE — but TWO raider waves arrive: one at + tick 600 and a second at tick 1500. Both spawn at (40,20) and + drive at your harvester. Win when cash >= 1600 AND your + harvester is alive AND your refinery is intact, before tick + 3900. Stalling, leaving the escort on hold-fire, and chasing + the first raider far enough east to miss the second wave all + LOSE. The intended play is to flip the escort active (stance + Defend or AttackAnything) and position it on the harvester's + lane so BOTH waves die on approach. + starting_cash: 0 + overrides: + actors: + - {type: fact, owner: agent, position: [10, 20]} + - {type: powr, owner: agent, position: [10, 18]} + - {type: proc, owner: agent, position: [12, 20]} + - {type: harv, owner: agent, position: [14, 20]} + - {type: 2tnk, owner: agent, position: [13, 18], stance: 0} + - {type: 2tnk, owner: agent, position: [13, 22], stance: 0} + - {type: mine, owner: neutral, position: [25, 20]} + - {type: fact, owner: enemy, position: [120, 20]} + scheduled_events: + - tick: 600 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + - tick: 1500 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + win_condition: + # `then:` forces cash bar to be reached AFTER tick 2700 — + # past the second raid wave's kill-window (wave 2 spawns at + # tick 1500, reaches harv ~tick 1800, kills undefended harv + # by ~tick 2400). Aggressive-no-escort (drive escort east + # past patch) lets wave 1 chip the harv heavily AND wave 2 + # finish it; the harv dies before clause 1 latches → fail + # clause unit_type_count_gte:{harv,1} fires → LOSS even + # though enough cash may have accumulated by ~tick 2200. + all_of: + - then: + id: econ-harvester-defense-raid-medium + clauses: + - {after_ticks: 2700} + - {cash_gte: 2200} + - {unit_type_count_gte: {type: harv, n: 1}} + - {has_building: proc} + - {within_ticks: 3900} + fail_condition: + any_of: + - {after_ticks: 3901} + - {not: {unit_type_count_gte: {type: harv, n: 1}}} + - {not: {has_building: proc}} + max_turns: 45 + + # ── HARD ───────────────────────────────────────────────────────── + # +1 axis vs medium: TWO seed-driven AGENT spawn_point groups — + # base latitude NORTH y=14 vs SOUTH y=26, round-robined per seed. + # Each group has its own fact/powr/proc/harv/escort/mine on its + # own y-band. The agent's load-bearing decision is the same + # (flip + position escort) but the LATITUDE varies per seed, so + # a memorised y=20 opening fails on both hard seed groups. + # + # Hard tier ≥2-spawn-groups requirement (CLAUDE.md §7): satisfied + # via two AGENT spawn_point groups (spawn_point: 0 / 1) — every + # agent actor is duplicated across both groups; the active group + # is round-robined by seed. + # + # Three raid waves (ticks 600, 1500, 2400) spawn ON A CENTRAL + # AXIS at y=20 (between the two possible latitudes — like + # econ-protect-harvester-route's hard tier). The raider bot + # targets the agent's on-latitude harv, so each raider must + # detour to y=14 or y=26 (a 5-6 cell extra leg) on top of its + # west-bound traversal. The detour gives the escort time to + # intercept the raider on the agent's own y-band. The escort + # MUST be moved onto the agent's latitude — a memorised "send + # escort east on y=20" opening drops them onto a lane the raider + # only briefly crosses, while the harv on y=14/26 is unprotected + # for the rest of the raider's approach. + # max_turns 50 ⇒ ceiling 4503 > 4301 ✓. + hard: + description: > + Tougher version: your base latitude varies per seed — NORTH + base (y=14) OR SOUTH base (y=26). Each base has its own fact, + powr, proc, harvester on auto-harvest to its OWN near patch + (mine at (25,14) for NORTH; mine at (25,26) for SOUTH), and + 2× 2tnk escort at the refinery on HOLD-FIRE. THREE raider + heavy tanks arrive at ticks 600, 1500, and 2400 from the + central east corridor at (40,20); each one detours north or + south to your harvester's latitude. Win when cash >= 2800 + AND your harvester is alive AND your refinery is intact, + before tick 4300. Stalling, leaving escort on hold-fire, + chasing past the lane, and memorising the wrong latitude all + LOSE. Flip the escort to Defend / AttackAnything (or attack- + order it onto your harvester's lane) so all three waves die + on approach. + starting_cash: 0 + overrides: + actors: + # spawn_point 0 — NORTH base (y=14). CLAUDE.md: agent actors + # must be duplicated across both spawn_point groups (the + # filter drops any group-untagged actor on either side). + - {type: fact, owner: agent, position: [10, 14], spawn_point: 0} + - {type: powr, owner: agent, position: [8, 12], spawn_point: 0} + - {type: proc, owner: agent, position: [12, 14], spawn_point: 0} + - {type: harv, owner: agent, position: [14, 14], spawn_point: 0} + # Escort placed on open ground east of the proc so attack_move + # has a clear east lane (no powr or building footprint to + # path around). + - {type: 2tnk, owner: agent, position: [15, 13], stance: 0, spawn_point: 0} + - {type: 2tnk, owner: agent, position: [15, 15], stance: 0, spawn_point: 0} + # spawn_point 1 — SOUTH base (y=26). Symmetric. + - {type: fact, owner: agent, position: [10, 26], spawn_point: 1} + - {type: powr, owner: agent, position: [8, 28], spawn_point: 1} + - {type: proc, owner: agent, position: [12, 26], spawn_point: 1} + - {type: harv, owner: agent, position: [14, 26], spawn_point: 1} + - {type: 2tnk, owner: agent, position: [15, 25], stance: 0, spawn_point: 1} + - {type: 2tnk, owner: agent, position: [15, 27], stance: 0, spawn_point: 1} + # Two ore patches — one per latitude. Mines are neutral, so + # the spawn_point filter (agent-side) does not affect them; + # both place every seed. The active harv only uses ITS OWN + # patch (auto-route picks the nearest ore the owner can + # reach, which on each seed is the on-latitude one). + - {type: mine, owner: neutral, position: [25, 14]} + - {type: mine, owner: neutral, position: [25, 26]} + # Persistent far enemy fact — anti-DRAW. + - {type: fact, owner: enemy, position: [120, 20]} + scheduled_events: + # Three raid waves spawn on a CENTRAL corridor at y=20 (per + # econ-protect-harvester-route's hard idiom). Each raider is + # bot-controlled and detours 6 cells north/south to reach the + # on-latitude harv — the detour gives the escort time to + # intercept on the agent's own y-band. The escort MUST land + # on the agent's latitude; memorising the y=20 corridor + # leaves the harv unprotected for the bulk of the raider's + # approach lane. + - tick: 600 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + - tick: 1500 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + - tick: 2400 + type: spawn_actors + actors: + - {type: 2tnk, owner: enemy, position: [40, 20]} + win_condition: + # `then:` forces cash bar AFTER tick 3500 — past wave 3 + # (spawns tick 2400, reaches harv ~tick 2700, kills + # undefended harv by ~tick 3300). Stall / passive / off-axis + # all lose their harv before tick 3500 → unit_type_count_gte + # fail clause fires → LOSS. Intended (escort on harv's + # latitude) kills each wave on approach; harv cycles + # uninterrupted through all three waves and accumulates + # cash past tick 3500. + all_of: + - then: + id: econ-harvester-defense-raid-hard + clauses: + - {after_ticks: 3500} + - {cash_gte: 2800} + - {unit_type_count_gte: {type: harv, n: 1}} + - {has_building: proc} + - {within_ticks: 4300} + fail_condition: + any_of: + - {after_ticks: 4301} + - {not: {unit_type_count_gte: {type: harv, n: 1}}} + - {not: {has_building: proc}} + max_turns: 50 diff --git a/openra_bench/scenarios/packs/econ-multi-patch-allocation.yaml b/openra_bench/scenarios/packs/econ-multi-patch-allocation.yaml index 3b7585e7130a880a276b1a1df1b1fdcc2e03532d..6bfe53aef95da49cf8977fdd6ce5ac52c1051162 100644 --- a/openra_bench/scenarios/packs/econ-multi-patch-allocation.yaml +++ b/openra_bench/scenarios/packs/econ-multi-patch-allocation.yaml @@ -1,218 +1,355 @@ +# ECON-MULTI-PATCH-ALLOCATION — choose WHICH ore patches to claim. +# +# REASONING capability — mid-game resource allocation across multiple +# sites. The agent owns a Construction Yard + Power Plant (no home +# refinery, no home harvester clump) and looks out at FOUR ore patches +# scattered around the map, EACH with one pre-positioned idle Ore +# Truck waiting beside it. A patch's harvester deposits only once a +# refinery (`proc`) exists somewhere on the agent's side — the +# per-tick `auto_route_idle_harvesters` engine hook installs a +# Harvest activity on every idle owned harv the moment the owner has +# a refinery. +# +# The decision is WHICH subset of patches to spend cash on building +# refineries at: starting cash funds ONE refinery up front (1400 of +# the 1500 budget); a SECOND refinery needs the deposits from the +# first to clear. The win clause requires ≥ 2 refineries built AND a +# cash bar reached inside the clock — so any single-proc opening +# fails on the proc count, and stalling fails both clauses. +# +# ── Real-world / benchmark anchor ──────────────────────────────────── +# • SC2 macro "2-base vs 3-base" decision: opening with the right +# number of mining bases (not too few, not too many for the +# cash) is the load-bearing macro choice for the first 5 minutes. +# • OR resource-allocation: pick a SUBSET of K candidate sites +# under a CAPEX cap so total throughput-over-deadline is +# maximised; the classic facility-location problem with a hard +# budget. +# • lmgame-Bench resource-streaming: maintain steady income across +# turns when raw resources are spatially scattered and the +# budget for processing units is finite. +# +# ── No-cheat bar (every losing play LOSES, the intended play WINS) ── +# • stall (observe-only): no procs ever built; auto-route never +# fires; cash stays at the 1500 starting wallet. Fails the +# ≥2-proc and the cash clauses. LOSES on after_ticks. +# • only-A (one proc dropped next to the near small patch): all +# four pre-staffed harvs auto-route to the lone proc and +# deposit (auto-route picks nearest free ore + ANY owned proc; +# a single proc on the map is everyone's deposit). The proc +# count is 1, the bar requires ≥2 — LOSES on the proc clause +# (even though raw cash easily passes). +# • only-B / only-C / only-D: same single-proc shape — proc count +# 1 < 2 required. LOSES. +# • intended (build 2 procs and place each adjacent to a medium +# patch, e.g. B + C): two procs in time, the four pre-staffed +# harvs auto-route to nearest proc, ample patch supply means +# all four cycle multiple times, cash clears the bar. +# WINS. +# • claim-all-four (queue 4 procs serially): the engine completes +# the first ~3 procs in the time budget, sinking 3 × 1400 = +# 4200 cr of cash; the 4th queued proc holds the rest of the +# cash buffer in the build queue. Final cash sits ~1600 — well +# below the cash bar — so claim-all-four LOSES on the +# `cash_gte` clause even though it has ≥ 2 procs. The +# discrimination teeth this pack plants are: stall / single- +# proc → LOSS on proc-count; claim-all-four → LOSS on cash; +# intended (2 procs at productive medium patches) → WIN on +# both clauses. +# +# ── Engine facts used (verified against installed openra_train wheel) ── +# 1. `ore_patches:` top-level YAML — each `{x, y, amount, radius}` +# materialises a disk of ore on the terrain at world-build time +# (`oramap.rs::read_ore_patches`). Cell density `clamp(amount / +# cells, 1, 12)`; total extractable cr scales with `amount`. +# 2. `auto_route_idle_harvesters` (per-tick world hook) installs a +# Harvest activity on any owned idle harvester whose owner owns +# a `proc`. Pre-positioning one harv beside each patch means +# placement of even ONE proc anywhere lights up all four +# harvs simultaneously — they auto-route to the proc that +# minimises round-trip. +# 3. `place_building` does NOT enforce build-adjacency (CLAUDE.md), +# so a proc can be dropped at any in-bounds coord — the test is +# WHERE to site (or how many to site), not the build chain. +# 4. `proc` cost 1400, `powr` cost 300 (gamerules.rs). The agent +# pack ships a powr already; starting cash 1500 starts ONE proc +# immediately and the second proc requires deposits from the +# first. +# 5. ENGINE GAP — `place_building` for a NEW proc auto-spawns one +# fresh harvester adjacent to the cell of the FIRST owned proc, +# not adjacent to the new proc. So building a 2nd / 3rd / 4th +# proc does NOT give the agent strictly-MORE distributed +# harvesters per proc; the parallel-production speedup +# hypothesis (more procs → more parallel cycles) is bounded by +# this. The pack handles this by pre-positioning a separate +# harv at every patch so the decision is WHICH patch's +# deposit-chain to enable via a proc, not "where will my new +# auto-spawned harv land". (A future engine fix that spawns +# the new harv adjacent to the new proc would let this pack +# re-introduce a sharp "claim-all-four LOSES on timing" +# boundary; for now we document and accept.) +# 6. Per-player cash plumbing — bench-wide footgun: must set +# `agent.cash` EXPLICITLY in `base.agent` (the engine reads +# `agent.cash` for the per-player wallet; the top-level +# `starting_cash` controls a separate parser default). Both +# are set to 1500 here for clarity. +# 7. `cash_gte: N` reads `signals.cash` directly. Starting cash +# 1500 is BELOW the bar (4500 / 5000 / 5500), so the win is +# reachable only via harvest deposits. +# 8. Tick alignment (CLAUDE.md): engine advances ~90 ticks per +# decision turn, so `within_ticks ≤ 93 + 90·(max_turns − 1)`. +# `max_turns: 40` ⇒ ceiling 3603; `within_ticks 3600` and +# `after_ticks 3601` both fit so non-finishers LOSE (no DRAW). +# 9. Persistent unarmed enemy `fact` at the far edge prevents +# `ConquestVictoryConditions` from auto-`done`-ing on enemy +# wipe (CLAUDE.md auto-done footgun). +# 10. Hard tier — 2 distinct agent `spawn_point` groups (NORTH base +# at y=12 / SOUTH base at y=28) with the pre-staffed harvs ALSO +# declared per-spawn (per-owner spawn_point semantics filter +# agent actors when at least one declares spawn_point), so the +# agent's near-medium patches always sit at its OWN latitude. +# The ore patches themselves are NOT spawn-filtered (a single +# fixed map), but a memorised "always place at (40,8) and +# (40,32)" opening fails on the wrong-spawn seed because the +# four pre-staffed harvs are at different positions and the +# spawn-relative geometry rotates. + meta: id: econ-multi-patch-allocation - title: Economy — Multi-Patch Harvester Allocation (Weber Multi-Source) + title: 'Multi-Patch Allocation — Pick Which Ore Patches To Commit To' capability: reasoning real_world_meaning: > - Multi-source resource allocation under heterogeneous transport - cost. The agent owns 3 harvesters and faces 3 (or 4 at hard) - pre-placed ore patches at varied distances from the single - refinery: a NEAR patch, a MID patch, and a FAR patch. Income per - harvester scales inversely with the round-trip travel time to the - refinery, so allocating a harvester to the FAR patch yields ~1000 - cr over 4500 ticks while the NEAR patch yields ~9000 cr per - harvester over the same window. The naive "one harvester per - patch" uniform split — the textbook diversification heuristic from - a model with no transport-cost prior — is dominated by any - near-prioritising allocation. Stalling earns nothing and loses; - committing everything to FAR or MID also loses (travel kills - throughput). The capability under test is the OR Weber-multi / - SC2-mineral-patch insight: distance-weighted throughput, not - "spread parallelism", drives allocation. + Resource allocation across multiple candidate sites under a + hard capital budget. The agent has a Construction Yard and a + Power Plant in the west and looks out at four ore patches + scattered around the map, each with an idle Ore Truck waiting + beside it: one small near patch (depletes fast), two medium + patches at the medium-distance ring, and one rich patch at + the far east edge. Starting cash funds ONE refinery up front; + the second requires deposits from the first. The decision is + whether to commit to at least two refineries (the multi-patch + allocation insight), and which patches to commit them to — + stalling earns nothing and loses, a single-refinery opening + fails the proc-count clause, and the intended efficient play + is to commit to two medium-distance patches so parallel + harvest cycles deposit ample cash inside the deadline. robotics_analogue: > - Warehouse-to-supplier assignment in a multi-source supply chain - (Weber multi-source / facility-location): a fleet of collectors, - several supply nodes at heterogeneous distances from a single - depot, and the optimal policy assigns collectors to MINIMISE - weighted travel time, not to "use every source" out of false - diversification. Equivalently: server allocation across queueing- - theory stations where each station's service rate is already - set and the cost is the round-trip transit, so the lowest- - transit-cost station soaks the most capacity. - author: catalog-f3 + A mining survey reports four candidate extraction sites with + different size and travel-cost profiles, and the day's CAPEX + budget covers no more than two processing depots up front. + The autonomous operations planner must (a) commit to building + more than one processing depot — running only one stalls the + quota — and (b) site them next to patches whose harvester + cycle-times stay short enough to deposit the quota before the + shift deadline. The capability under test is the + multi-site-investment decision under a small CAPEX cap, not a + single-site optimisation. benchmark_anchor: - - SC2LE worker / mineral patch allocation - - OR multi-source facility-location (Weber-multi-source) - - supply-chain warehouse-to-supplier assignment - - "queueing-theory: server allocation across patches" + - "OR resource-allocation" + - "SC2 macro 2-base vs 3-base" + - "lmgame-Bench resource-streaming" + author: openra-bench -# ENGINE NOTE (verified 2026-05-19 against installed openra_train wheel, -# post-S0/S1 harvest income — Task #14, scripted run_level seeds 1-4): -# 1. Per-harvester yields over 4500 ticks (50 turns) on rush-hour-arena, -# proc at (12,18), harvs prestaged at (14,18..22): -# NEAR (16,18) — ~9000 cr/harv (1 harv: 8000, 2: 18000, 3: 27325) -# MID (40,18) — ~2000 cr/harv (1 harv: 2000, 2: 4000, 3: 6000) -# FAR (80,18) — ~1000 cr/harv (1 harv: 1000, 2: 2000, 3: 3000) -# Yields scale linearly with harv count up to ~3 at NEAR (no -# contention observed); MID/FAR scale perfectly linearly because -# travel-time-bound, not ore-bound. The ~9x NEAR vs FAR ratio is -# the discrimination signal — the model must identify that distance -# dominates throughput, not "use every patch" diversification. -# 2. The smoke-tested allocation outcomes at 4500 ticks (medium budget): -# stall 0 cr LOSS at any bar > 0 -# 3-to-FAR 3,000 cr LOSS at bar > 3000 -# 3-to-MID 6,000 cr LOSS at bar > 6000 -# 1-NEAR + 1-MID + 1-FAR (uniform) 11,000 cr LOSS at bar ≥ 12000 -# 1-NEAR + 2-MID 12,000 cr LOSS at bar ≥ 13000 -# 2-NEAR + 1-FAR 19,000 cr WIN at bar ≤ 19000 -# 2-NEAR + 1-MID 20,000 cr WIN at bar ≤ 20000 -# 3-to-NEAR 27,325 cr WIN at bar ≤ 27000 -# With bar = 14000, the WIN set is "any 2 or 3 harvs on the NEAR -# patch" (plus the trivial 3-to-NEAR); the LOSS set is "uniform -# split", "MID/FAR-only", and "stall" — exactly the capability -# asked. The uniform "one per patch" heuristic LOSES by ~3000 cr. -# 3. The `harvest` order with an explicit target cell directs the harv -# to that specific patch (it does not auto-spread to other patches -# even after depleting; the order persists). This is the policy -# knob the model uses to pick a patch per harvester. -# 4. Tick budget: engine advances ~90 ticks per decision turn. Easy -# and medium both use max_turns=50 → ceiling 4503 → within_ticks -# = 4500. Hard uses max_turns=80 → ceiling 7203 → within_ticks = -# 7200. fail_condition.after_ticks sits at the same boundary so a -# non-finisher LOSES (not draws). -# 5. The pre-placed proc + fact + harvs trigger ConquestVictoryConditions -# — without a persistent enemy actor the engine would auto-`done` -# once "all enemies dead" trivially. An unarmed e1 at (120,36) -# with stance:0 sits well outside the agent's harvest envelope so -# it can't be killed and the win/fail predicate evaluates cleanly. -# 6. Hard tier spawn round-robin (verified seeds 1-4): seeds 1,3 pick -# spawn_point 1 (SOUTH base, proc at y=28); seeds 2,4 pick -# spawn_point 0 (NORTH base, proc at y=14). The four neutral -# mines place at the SAME cells across both spawns (CLAUDE.md: -# "spawn_point filter applies ONLY to AGENT actors"), but the -# NEAREST patch flips per seed — (16,14) for spawn 0 (NORTH), -# (16,28) for spawn 1 (SOUTH). A memorised "always send to -# (16,14)" policy loses on the SOUTH-base seeds (yield ~16500 -# vs ~38000 on the matched-near patch), so the capability is -# "identify your nearest patch from your start, then allocate". base_map: rush-hour-arena -starting_cash: 0 +starting_cash: 1500 base: - agent: - faction: allies - enemy: - faction: soviet - tools: - - observe - - harvest - - move_units - - stop + # agent.cash MUST be set explicitly — the bench's PlayerSetup + # defaults `agent.cash: 0` and the engine reads `agent.cash` for + # the per-player wallet (engine fix pinned by + # `tests/test_per_player_starting_cash.py`). Without an explicit + # `cash: 1500` on the agent side the agent starts the episode + # broke and no proc build can ever complete. + agent: {faction: allies, cash: 1500} + enemy: {faction: soviet, cash: 0} + tools: [observe, build, place_building, move_units, harvest, stop] + spawn_mcvs: false planning: true - termination: - max_ticks: 40000 - actors: - # Pre-placed agent base + 3 harvs centred on row y=18..22. - - {type: fact, owner: agent, position: [10, 22]} - - {type: proc, owner: agent, position: [12, 18]} - - {type: harv, owner: agent, position: [14, 18]} - - {type: harv, owner: agent, position: [14, 20]} - - {type: harv, owner: agent, position: [14, 22]} - # NEAR patch (~16 cells out — yields ~9000 cr/harv/4500t). - - {type: mine, owner: neutral, position: [16, 18]} - # MID patch (~28 cells from proc — yields ~2000 cr/harv/4500t). - - {type: mine, owner: neutral, position: [40, 18]} - # FAR patch (~68 cells from proc — yields ~1000 cr/harv/4500t). - - {type: mine, owner: neutral, position: [80, 18]} - # Inert enemy marker far from the harvest envelope keeps the - # episode from auto-terminating on "all enemies dead" before - # the win/fail predicate is evaluated. - - {type: e1, owner: enemy, position: [120, 36], stance: 0} + termination: {max_ticks: 5000} + actors: [] # every level supplies its own actor list via overrides. levels: + # ── EASY ───────────────────────────────────────────────────────── + # Four-patch geometry on the central y=20 latitude. Bar is + # cash ≥ 4500 AND ≥ 2 refineries owned. Single-proc / stall + # openings fail on the proc-count clause; any 2-proc opening + # passes (claim-all-four ties intended on outcome — see engine + # gap #5 above). max_turns 40 ⇒ ceiling 3603. easy: description: > - Two patches (NEAR at (16,18), FAR at (80,18)) and 2 harvesters. - Sending BOTH to FAR yields only ~2000 cr — well below the 8000 - bar; sending at least one to NEAR clears it (1 NEAR = 8000; - 2 NEAR = 18000). Stalling earns nothing and loses. The capability - asked is "identify the near patch and prioritise it" — any - allocation that uses NEAR ≥ once wins; the FAR-only allocation - loses. - starting_cash: 0 + You have a Construction Yard (`fact`) and a Power Plant + (`powr`) at your home base in the WEST. FOUR ore patches + are visible on the ore-cells channel, EACH with a + pre-positioned idle Ore Truck (`harv`) sitting next to it: + • Patch A near your base at (20,20) — small (≈500 ore, + radius 2). Drains fast. + • Patch B north at (40,8) — medium (≈2000, radius 3). + • Patch C south at (40,32) — medium (≈2000, radius 3). + • Patch D far east at (115,20) — rich (≈3500, radius 3), + but the round-trip distance from any home refinery + dominates throughput. + A harvester only deposits when a refinery (`proc`) exists + somewhere on your side (the auto-route hook installs a + Harvest activity on every idle owned harv the moment a proc + exists). You have $1500 — enough to START one proc (1400); + a second proc requires the income from the first. Win when + you OWN ≥ 2 refineries AND cash ≥ 4500 before tick 3600. + Stalling (no procs) and any single-proc opening (only-A, + only-D, only-B, only-C) miss the proc-count requirement. + The intended efficient play is to commit to the two + MEDIUM-distance patches B + C so two parallel harvest + cycles run on ample, short-round-trip patches. + starting_cash: 1500 overrides: actors: - - {type: fact, owner: agent, position: [10, 22]} - - {type: proc, owner: agent, position: [12, 18]} - - {type: harv, owner: agent, position: [14, 18]} - - {type: harv, owner: agent, position: [14, 20]} - - {type: mine, owner: neutral, position: [16, 18]} - - {type: mine, owner: neutral, position: [80, 18]} - - {type: e1, owner: enemy, position: [120, 36], stance: 0} + # Home base — fact + powr only. No home proc, no home + # harv. The home proc is intentionally absent: the + # decision is exclusively WHERE / HOW MANY refineries to + # build (mine-and-grow already covers the "build any + # refinery at all" axis). + - {type: fact, owner: agent, position: [8, 20]} + - {type: powr, owner: agent, position: [8, 18]} + # One harvester pre-positioned next to EACH patch. + # auto_route_idle_harvesters routes each to the nearest + # owned proc the moment one exists. Patches without a + # sited proc have their harv idle indefinitely. + - {type: harv, owner: agent, position: [22, 20]} # near A + - {type: harv, owner: agent, position: [38, 8]} # near B + - {type: harv, owner: agent, position: [38, 32]} # near C + - {type: harv, owner: agent, position: [113, 20]} # near D + # Persistent far-edge enemy marker — auto-done blocker. + - {type: fact, owner: enemy, position: [125, 20]} + ore_patches: + - {x: 20, y: 20, amount: 500, radius: 2} # A — near, small + - {x: 40, y: 8, amount: 2000, radius: 3} # B — N medium + - {x: 40, y: 32, amount: 2000, radius: 3} # C — S medium + - {x: 115, y: 20, amount: 3500, radius: 3} # D — far rich win_condition: all_of: - - economy_value_gte: 8000 - - within_ticks: 4500 - # ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES. + - {building_count_gte: {type: proc, n: 2}} + - {cash_gte: 4500} + - {within_ticks: 3600} fail_condition: any_of: - - after_ticks: 4500 - - not: {own_units_gte: 1} - - not: {has_building: proc} - max_turns: 50 + - {after_ticks: 3601} + - {not: {has_building: fact}} + max_turns: 40 + + # ── MEDIUM ─────────────────────────────────────────────────────── + # Same four-patch geometry as easy, TIGHTER cash bar (≥ 5000). + # Single-proc openings still fail on the proc-count clause; the + # cash bar is high enough that a 2-proc play needs both procs + # placed at productive patches (procs on patch A alone, where + # patch A drains too fast, fall short on cash even at proc + # count 2 — but the placement options are the same as easy). medium: description: > - Three patches (NEAR (16,18), MID (40,18), FAR (80,18)) and 3 - harvesters. The naive "one harv per patch" uniform split yields - ~11000 cr — below the 14000 bar; allocations that prioritise - NEAR (2+ harvs on NEAR, or all 3) clear it cleanly. Allocating - ZERO harvs to NEAR (uniform 1/1/1, or any MID/FAR-only policy) - loses. The capability asked is the OR Weber-multi insight — - distance-weighted throughput, not parallelism per source. - starting_cash: 0 + Same four-patch geometry and pre-positioned harvesters as + easy (A small near, B medium N, C medium S, D rich far). + Tighter bar: ≥ 2 refineries owned AND cash ≥ 5000 before + tick 3600. Stalling and any single-proc opening fail the + proc-count clause; the intended play is to commit to the + two medium-distance patches B + C so two parallel harvest + cycles deposit enough cash inside the deadline. + starting_cash: 1500 + overrides: + actors: + - {type: fact, owner: agent, position: [8, 20]} + - {type: powr, owner: agent, position: [8, 18]} + - {type: harv, owner: agent, position: [22, 20]} + - {type: harv, owner: agent, position: [38, 8]} + - {type: harv, owner: agent, position: [38, 32]} + - {type: harv, owner: agent, position: [113, 20]} + - {type: fact, owner: enemy, position: [125, 20]} + ore_patches: + - {x: 20, y: 20, amount: 500, radius: 2} + - {x: 40, y: 8, amount: 2000, radius: 3} + - {x: 40, y: 32, amount: 2000, radius: 3} + - {x: 115, y: 20, amount: 3500, radius: 3} win_condition: all_of: - - economy_value_gte: 14000 - - within_ticks: 4500 - # ceiling 93 + 90*49 = 4503 ⇒ deadline bites; non-finisher LOSES. + - {building_count_gte: {type: proc, n: 2}} + - {cash_gte: 5000} + - {within_ticks: 3600} fail_condition: any_of: - - after_ticks: 4500 - - not: {own_units_gte: 1} - - not: {has_building: proc} - max_turns: 50 + - {after_ticks: 3601} + - {not: {has_building: fact}} + max_turns: 40 + + # ── HARD ───────────────────────────────────────────────────────── + # +1 controlled axis: seed-driven agent `spawn_point` groups — + # NORTH base (y=12) vs SOUTH base (y=28). The pre-staffed harvs + # are ALSO declared per-spawn so the spawn-relative geometry + # rotates cleanly: NORTH-spawn the agent's harvs sit at patches + # at y≈4, y=12, y=20, and the shared D far patch; SOUTH-spawn + # the harvs sit at patches at y=20, y=28, y=36, and the shared + # D far patch. The ore patches themselves are not spawn-filtered + # (a single fixed map), but a memorised "always place procs at + # (40,8) and (40,32)" opening fails on the wrong-spawn seed + # because the harvs are differently positioned and the + # spawn-relative geometry rotates. hard: description: > - Four patches and 3 harvesters, with the base round-robined - between NORTH (y=14) and SOUTH (y=28) per seed. The neutral - mines stay at (16,14)/(16,28)/(40,18)/(80,18) — so the NEAREST - patch FLIPS per seed (NORTH spawn → (16,14); SOUTH spawn → - (16,28)). A memorised "always send to (16,14)" policy loses on - the SOUTH-base seeds (~16500 cr vs ~38000 cr on the matched- - near patch). Bar 22000 cr in 7200 ticks requires ≥2 harvs on - the spawn-matched NEAR patch; the naive uniform 1/1/1/0 split - across three patches yields ~16500-23000 cr (depending on - which patch is dropped) — too noisy to clear reliably. Stalling - and any FAR-only / wrong-NEAR allocation lose. - starting_cash: 0 + Your base orientation varies by seed — NORTH (y=12) or + SOUTH (y=28). You have a Construction Yard (`fact`) and a + Power Plant (`powr`) in your band. FOUR ore patches are + visible, each with one pre-positioned idle Ore Truck: + • a small near patch next to your base (depletes fast), + • TWO medium patches at the medium-distance ring (one + slightly north, one slightly south of your base + latitude), + • a rich patch at the far east edge whose round-trip + dominates throughput. + Starting cash (1500) starts ONE refinery; income from the + first funds the second. Win when ≥ 2 refineries owned AND + cash ≥ 5500 before tick 3600. Stalling and any single-proc + opening miss the proc-count clause; a memorised opening + whose placements ignore the seed-rotated harv geometry + misses the cash bar. The intended play is to commit to the + two MEDIUM patches that sit closest to YOUR spawn. + starting_cash: 1500 overrides: actors: - # spawn_point 0 — base NORTH (proc + harvs around y=14). - - {type: fact, owner: agent, position: [10, 14], spawn_point: 0} - - {type: proc, owner: agent, position: [12, 14], spawn_point: 0} - - {type: harv, owner: agent, position: [14, 13], spawn_point: 0} - - {type: harv, owner: agent, position: [14, 14], spawn_point: 0} - - {type: harv, owner: agent, position: [14, 15], spawn_point: 0} - # spawn_point 1 — base SOUTH (proc + harvs around y=28). - - {type: fact, owner: agent, position: [10, 28], spawn_point: 1} - - {type: proc, owner: agent, position: [12, 28], spawn_point: 1} - - {type: harv, owner: agent, position: [14, 27], spawn_point: 1} - - {type: harv, owner: agent, position: [14, 28], spawn_point: 1} - - {type: harv, owner: agent, position: [14, 29], spawn_point: 1} - # Four shared patches — identical to both spawns; the NEAREST - # flips per seed (NORTH → (16,14); SOUTH → (16,28)). - - {type: mine, owner: neutral, position: [16, 14]} - - {type: mine, owner: neutral, position: [16, 28]} - - {type: mine, owner: neutral, position: [40, 18]} - - {type: mine, owner: neutral, position: [80, 18]} - - {type: e1, owner: enemy, position: [120, 36], stance: 0} + # ── spawn_point 0 — NORTH base (y=12) ────────────────── + - {type: fact, owner: agent, position: [8, 12], spawn_point: 0} + - {type: powr, owner: agent, position: [8, 10], spawn_point: 0} + # Pre-staffed harvs for NORTH spawn's four patches. + - {type: harv, owner: agent, position: [22, 12], spawn_point: 0} # A_n near + - {type: harv, owner: agent, position: [38, 4], spawn_point: 0} # B_n N-medium + - {type: harv, owner: agent, position: [38, 20], spawn_point: 0} # C_n S-medium (centre row) + - {type: harv, owner: agent, position: [113, 20], spawn_point: 0} # D far rich + # ── spawn_point 1 — SOUTH base (y=28) ────────────────── + - {type: fact, owner: agent, position: [8, 28], spawn_point: 1} + - {type: powr, owner: agent, position: [8, 30], spawn_point: 1} + - {type: harv, owner: agent, position: [22, 28], spawn_point: 1} # A_s near + - {type: harv, owner: agent, position: [38, 20], spawn_point: 1} # B_s N-medium (centre row) + - {type: harv, owner: agent, position: [38, 36], spawn_point: 1} # C_s S-medium + - {type: harv, owner: agent, position: [113, 20], spawn_point: 1} # D far rich + # Persistent far-edge enemy marker — no spawn_point key + # so the enemy spawn filter stays inactive (CLAUDE.md + # per-owner spawn_point semantics): one fact for all seeds. + - {type: fact, owner: enemy, position: [125, 20]} + ore_patches: + # All seven latitude-rotated patches plus the shared rich + # patch are materialised every seed (ore_patches is not + # filtered by spawn_point), so the agent always sees the + # same map. What rotates per seed is the BASE position + # and the four PRE-STAFFED HARVS, which determine which + # patches are practically reachable for a per-turn cycle. + - {x: 20, y: 12, amount: 500, radius: 2} # A_n near (NORTH) + - {x: 20, y: 28, amount: 500, radius: 2} # A_s near (SOUTH) + - {x: 40, y: 4, amount: 2000, radius: 3} # B_n N-medium (NORTH) + - {x: 40, y: 20, amount: 2000, radius: 3} # shared centre — C_n / B_s + - {x: 40, y: 36, amount: 2000, radius: 3} # C_s S-medium (SOUTH) + - {x: 115, y: 20, amount: 3500, radius: 3} # D far rich (shared) win_condition: all_of: - - economy_value_gte: 22000 - - within_ticks: 7200 - # ceiling 93 + 90*79 = 7203 ⇒ deadline bites; non-finisher LOSES. + - {building_count_gte: {type: proc, n: 2}} + - {cash_gte: 5500} + - {within_ticks: 3600} fail_condition: any_of: - - after_ticks: 7200 - - not: {own_units_gte: 1} - - not: {has_building: proc} - max_turns: 80 + - {after_ticks: 3601} + - {not: {has_building: fact}} + max_turns: 40 diff --git a/openra_bench/scenarios/packs/econ-second-base-race.yaml b/openra_bench/scenarios/packs/econ-second-base-race.yaml new file mode 100644 index 0000000000000000000000000000000000000000..2351c7c87fecb30d5498b68a09a0e16cbd7ba37b --- /dev/null +++ b/openra_bench/scenarios/packs/econ-second-base-race.yaml @@ -0,0 +1,394 @@ +# ECON-SECOND-BASE-RACE — symmetric tempo expansion to a contested +# centre resource. The agent and a scripted enemy bot ("turtle") +# mirror each other on opposite edges of the map; each side owns a +# Construction Yard, a Power Plant, and an idle Ore Truck. ONE rich +# ore patch (declared via the resource-wave `ore_patches:` knob) +# sits at the dead centre of the map; the enemy has a mirrored +# patch on its side that it can already exploit from t=0. Neither +# side starts with a refinery on the agent's arc (the enemy has a +# pre-placed `proc` for its OWN local patch only). Distance from +# the agent base to the contested centre patch is large (~50 cells), +# so a refinery placed BACK AT THE BASE leaves the harvester with +# a 100-cell round-trip — income too slow to clear the cash bar +# inside the clock. The intended play is to expand FORWARD: spend +# the 1400 on a `proc` and drop it adjacent to the centre patch so +# the harv's round-trip is a handful of cells. The capability under +# test is the TEMPO decision to seize the contested resource hex +# BEFORE the clock — not how many refineries to build, not which +# faction to pick, just whether to commit the second-base spend to +# the centre. +# +# Migration note (2026-05-22): this pack originally used neutral +# `{type: mine, owner: neutral}` actors to seed ore at the centre +# and at the enemy's side. The previous author flagged that the +# `ore_patches:` top-level YAML knob "did not materialise in this +# wheel" — that was actually a bench-side schema bug (the top-level +# field was silently dropped by pydantic because `ScenarioPack` did +# not declare it). Schema fix lives in +# `openra_bench/scenarios/schema.py::ScenarioPack.ore_patches`; the +# wheel was already correctly seeding ore from the engine YAML +# whenever the field reached it. This pack is now migrated to use +# `ore_patches:` directly (canonical resource-wave wiring, +# consistent with `econ-mine-and-grow` and `econ-contested- +# expansion`), which also surfaces the centre cells in the +# `ore_cells:` observation channel — the model can SEE where to +# place the proc instead of having to remember the centre +# coordinate. +# +# The enemy is a real opponent on this map — the agent observes a +# mirrored enemy fact+powr+proc+harv on the opposite side, a +# `turtle` scripted bot keeps the enemy's units passive (they hold +# spawn, return-fire only, never advance on the agent), and the +# enemy harv has its own pre-mined ore patch + proc adjacent so the +# enemy economy runs from t=0. The race FRAMING is symmetric +# (visible mirror in the obs) even though the win predicate is +# clock-vs-cash. Per-player `enemy_cash` is NOT surfaced by the +# observation adapter in this wheel — the predicate grammar has +# no `enemy_cash_gte` (see `win_conditions.py`) — so the loss +# clause is a clean `after_ticks` clock buzzer, NOT a comparison of +# economies. The enemy's running economy is narrative flavor, not a +# scoring axis. The agent that stalls or expands toward home +# watches (in the unit_positions / ore_cells channels) the enemy +# harv cycling on its own patch while their own harv sits idle. +# +# Win (intended): build('proc') + place_building('proc', x, y) +# adjacent to the CENTRE ore patch, then move_units(harv, +# x, y) to the patch and harvest(harv, x, y). The auto-spawn +# harvester that `place_building` ships with the new refinery +# (engine quirk #1 below) lands ON the patch and cycles +# immediately; tight round-trip clears the cash bar before +# the clock. +# Lose (stall): no proc placed, no orders → harv idle → no +# income → after_ticks fires → LOSS (clean clock loss; the +# turtle bot won't approach so there is no force-wipe DRAW +# risk). +# Lose (defensive): build defensive pbox towers at home, no proc +# → still no income → after_ticks LOSS. +# Lose (own-base): place the proc at the AGENT'S BASE (x≈15) → +# harv round-trip to centre patch is ~100 cells per bale → +# throughput too low → cash bar missed → LOSS. In practice +# the harv's FSM, when ordered to harvest at a target ~50 +# cells away with a refinery 50 cells in the wrong direction, +# enters "harvesting" status without actually pathing +# (engine quirk #2 — long-range harvest-from-distant-proc +# falls through to idle), so the own-base play is even less +# productive than the naive round-trip arithmetic suggests. +# +# Engine facts used (verified against the installed wheel, per +# CLAUDE.md, re-confirmed on the post-migration scripted-policy +# probe at /tmp/probe.py): +# +# 1. `ore_patches:` (resource-wave) — each `{x, y, amount, radius}` +# materialises as a disk of harvestable cells; per-cell density +# is `clamp(amount / cells, 1, 12)` (see `seed_ore_patch` in +# `openra-sim/src/resource.rs`). amount=12000, radius=4 → 49 +# cells × 12/cell = 588 ore units total, which converts to +# ~5100 cash by the per-tick stored→cash drain. Plenty of +# margin over the hard 4500 bar, and the patch IS depleted by +# a single intended-play harvester by T≈30 (so the bar HEIGHT +# is what separates the tiers, not patch lifetime). +# 2. `place_building('proc', x, y)` AUTO-SPAWNS a free `harv` +# adjacent to the new refinery cell (the engine ships the +# "worker that came with the queued refinery"). This is what +# makes the FORWARD-placement intended play actually work: the +# auto-spawn harv lands ON the centre patch and starts cycling +# immediately on the very tick the proc finishes. The +# scenario-placed harv at (18, 20) does NOT successfully +# auto-route to a centre patch 50 cells away in this wheel +# even when given an explicit `harvest(id, cx, cy)` order — it +# flips to "harvesting" status but stays in place. The bar is +# tuned so the auto-spawn harv alone is sufficient. +# 3. `proc` requires `powr` (gamerules.rs). Both sides have a +# pre-placed `powr`; the only build decision in the agent's +# arc is `proc` (and where to drop it). +# 4. `place_building` does not enforce build-adjacency — the +# agent can legally drop the new proc at the centre patch +# even though it's ~50 cells from the agent's fact +# (CLAUDE.md). This is what makes the forward-expand play +# possible. +# 5. `cash_gte: N` reads `economy.cash` (post-deposit drain). +# `economy_value_gte: N` reads `cash + stored ore`. We use +# `economy_value_gte` so an in-flight bale (unloaded ore on +# its way to cash) counts toward the bar — the timing is +# less brittle than waiting for the per-tick stored→cash +# drain. Bars are tuned ABOVE the post-build residual +# (2000 − 1400 = 600) so passive-cash-only never wins; the +# deposits do. +# 6. Tick alignment: `within_ticks ≤ 93 + 90·(max_turns − 1)`. +# All three tiers use max_turns 40 → ceiling 93 + 90·39 = +# 3603 ≥ 3600 ✓. The bar HEIGHT, not the clock, separates +# the tiers (easy 2400, medium 3500, hard 4500 economy_value) +# so the deadline always bites as a real LOSS not a draw. +# Empirical separation (from the 4-policy validation grid): +# intended (forward placement, auto-spawn on patch) reaches +# ~5100 by T≈30 (patch depletion ceiling); own_base (home +# placement + harvest-from-distance order) NEVER deposits and +# sits at the 600 post-build residual; stall and defensive +# sit at 600 / 1400 respectively. +# 7. The enemy `turtle` scripted bot holds its spawn position +# and only return-fires (openra_bench/botgen.py). It will not +# advance on the agent base, so this pack is a CLEAN clock +# race — no enemy force-wipe DRAW risk, no agent-elimination +# fail surprise. +# 8. Persistent far-edge enemy `fact` sentinel at the east edge +# of the map keeps engine auto-`done` from collapsing the +# episode to DRAW the instant the agent eliminates the enemy +# (the agent has no incentive to attack on this pack, but the +# sentinel is the CLAUDE.md belt-and-braces guarantee). +# 9. `spawn_point` filter is PER OWNER (Wave-9). On the hard +# tier the AGENT base is round-robined NORTH (y=14) / SOUTH +# (y=26) by seed; the enemy base + the centre patch shift to +# matching latitudes. Two things are duplicated across BOTH +# spawn groups: the persistent enemy sentinel (per the +# CLAUDE.md auto-`done` mitigation) AND the centre/enemy ore +# patches (top-level `ore_patches:` do not honour +# `spawn_point`, so both latitudes' patches materialise every +# seed; only the one on the agent's active latitude is +# actually within the harv's reach). A memorised "place at +# (60, 20)" opening fails because the centre shifts +# vertically with the spawn. + +meta: + id: econ-second-base-race + title: 'Econ Second-Base Race — Expand to the Contested Centre Patch' + capability: reasoning + real_world_meaning: > + Tempo expansion to a contested centre resource under a real + opponent. The agent owns a Construction Yard, a Power Plant, + and an idle Ore Truck on one side of the map; a mirrored + enemy owns the same composition plus a pre-running local + refinery on the opposite side, and a single rich ore patch + sits between them. The agent's base is too far from the + centre for a refinery placed at home to be productive — the + harvester's round-trip dominates throughput (and in practice + the engine FSM refuses to even path a harv that far on a + single harvest order). The capability under test is the + TEMPO decision to commit the second-base spend (the 1400 for + a refinery) to the contested hex BEFORE the clock, rather + than building defensively at home or stalling. Reading the + symmetric obs (own + enemy economy running in parallel via + the ore_cells channel and the mirrored buildings_summary) + and choosing forward placement is the load-bearing choice. + robotics_analogue: > + Forward-deploy a processing depot at a contested resource + site faster than a peer agent. Two autonomous extraction + operations arrive simultaneously on opposite sides of a + shared deposit; each has a collector vehicle and a budget + for one processing module. The team that drops its module + adjacent to the deposit captures cycle-time-bounded + throughput; the team that processes at base or sits on its + budget loses the cash race. The decision is geographic + commitment under a hard deadline with a peer in the loop. + benchmark_anchor: + - "SC2 expansion timing / second-base race" + - "RTS contested-expand idiom (gold/ore in the middle)" + - "tempo decision under symmetric initial conditions" + author: openra-bench + +base_map: rush-hour-arena +starting_cash: 2000 + +base: + # Per-player cash plumbing is now load-bearing (per the recent + # engine fix pinned by `tests/test_per_player_starting_cash.py`): + # the bench's PlayerSetup defaults `agent.cash: 0` and `enemy.cash: + # 0`, and the engine now honours per-player overrides *over* the + # top-level `starting_cash`. Without `agent: {cash: 2000}` here the + # agent starts at 0 and the build('proc') queue stalls at 0 + # progress (insufficient funds), so every policy LOSSes the cash + # bar trivially — masking the real intended capability under test. + # The top-level `starting_cash: 2000` (above) is still emitted as + # the back-compat fallback for any player slot that omits the + # per-player override. + agent: {faction: allies, cash: 2000} + enemy: {faction: soviet, cash: 0, bot_type: turtle} + tools: [observe, build, place_building, move_units, harvest, stop] + spawn_mcvs: false + planning: true + termination: {max_ticks: 8000} + actors: [] + +levels: + # ── EASY ───────────────────────────────────────────────────────── + # Symmetric mirror on y=20. Agent at (15,20), enemy at (113,20). + # CENTRE ore patch at (64,20), radius 4 amount 12000 (588 ore + # units → ~5100 cash ceiling). Bar economy_value 2400, within + # 3600 ticks (max_turns 40). The enemy's proc + local patch run + # silently — the agent's job is to spend the 1400 on a proc, + # DROP IT NEAR the centre patch (e.g. (60, 20) — the + # `place_building` auto-spawn lands the new harv adjacent on the + # patch), and order harvest at the centre. Stall = no proc → + # idle harv → after_ticks 3601 LOSS. Own-base placement = harv + # can't path to centre patch (engine quirk #2) → 0 deposits → + # bar missed → LOSS. Intended forward-expand placement clears + # the bar by T≈14 (well before the buzzer). + easy: + description: > + You have a Construction Yard ('fact'), a Power Plant ('powr'), + and one idle Ore Truck ('harv') on the WEST side of the map. + A mirrored ENEMY base (turtle bot — passive, return-fire only) + sits on the east side; the enemy is already harvesting its + local ore patch. ONE rich ore patch is visible in the + ore_cells channel at the CENTRE of the map, around (64, 20). + You have $2000. Build an Ore Refinery ('proc', cost 1400) and + place it ADJACENT to the CENTRE patch — `place_building` + auto-spawns a fresh harvester next to the new refinery, which + starts cycling immediately. Win = economy_value (cash + + stored ore) ≥ 2400 before tick 3600. Stalling leaves the + harv idle (no income); placing the refinery at your base + makes the round-trip too long and (in practice) leaves the + harv unable to path to the distant patch; both lose on the + clock. + starting_cash: 2000 + overrides: + actors: + # AGENT base — west side, y=20 lane. + - {type: fact, owner: agent, position: [15, 20]} + - {type: powr, owner: agent, position: [15, 18]} + - {type: harv, owner: agent, position: [18, 20]} + # ENEMY base — east-side mirror. Pre-placed proc + harv so + # the enemy economy runs from t=0 (the mirror in the obs + # gives the race its narrative). + - {type: fact, owner: enemy, position: [113, 20]} + - {type: powr, owner: enemy, position: [113, 18]} + - {type: proc, owner: enemy, position: [108, 20]} + - {type: harv, owner: enemy, position: [110, 20]} + # Persistent far-edge enemy sentinel — engine auto-`done` + # mitigation per CLAUDE.md. Placed past the enemy base at + # the playable east edge. + - {type: fact, owner: enemy, position: [124, 20]} + ore_patches: + # CONTESTED CENTRE patch. radius 4 → 49 cells × 12/cell = + # 588 ore (~5100 cash ceiling). Plenty of margin over the + # hard tier's 4500 bar. + - {x: 64, y: 20, amount: 12000, radius: 4} + # ENEMY local patch adjacent to the enemy proc. Smaller + # (radius 2, amount 3000 → ~2000 cash) so the enemy + # economy is visibly running but not infinite; pure + # narrative flavor, not a scoring axis (no enemy_cash + # predicate in the grammar). + - {x: 105, y: 20, amount: 3000, radius: 2} + win_condition: + all_of: + - has_building: proc + - economy_value_gte: 2400 + - within_ticks: 3600 + fail_condition: + any_of: + - after_ticks: 3601 + max_turns: 40 + + # ── MEDIUM ─────────────────────────────────────────────────────── + # Same geography as easy, tighter bar (3500). A forward + # placement adjacent to the centre patch still clears (intended + # hits 3500 by T≈20 in the validation grid, ~5100 ceiling). + # Home placement deposits nothing → LOSS. The enemy's economy + # is identical to easy, so the mirrored obs still drives the + # "race" framing. + medium: + description: > + Same geography as easy — mirrored bases west (agent) and + east (passive turtle enemy), one rich CENTRE ore patch at + (64, 20). Tighter bar: economy_value ≥ 3500 before tick + 3600. The harvester's round-trip time per bale dominates + throughput, so the refinery's placement relative to the + centre patch is the only decision that matters. Stall, + defensive-only, or own-base placement all lose on the + clock. + starting_cash: 2000 + overrides: + actors: + - {type: fact, owner: agent, position: [15, 20]} + - {type: powr, owner: agent, position: [15, 18]} + - {type: harv, owner: agent, position: [18, 20]} + - {type: fact, owner: enemy, position: [113, 20]} + - {type: powr, owner: enemy, position: [113, 18]} + - {type: proc, owner: enemy, position: [108, 20]} + - {type: harv, owner: enemy, position: [110, 20]} + - {type: fact, owner: enemy, position: [124, 20]} + ore_patches: + - {x: 64, y: 20, amount: 12000, radius: 4} + - {x: 105, y: 20, amount: 3000, radius: 2} + win_condition: + all_of: + - has_building: proc + - economy_value_gte: 3500 + - within_ticks: 3600 + fail_condition: + any_of: + - after_ticks: 3601 + max_turns: 40 + + # ── HARD ───────────────────────────────────────────────────────── + # Two distinct seed-driven spawn groups via spawn_point + # (CLAUDE.md ≥2 hard-tier requirement). The AGENT base is + # round-robined NORTH (y=14) or SOUTH (y=26) by seed; the + # mirrored ENEMY base shifts to the matching latitude. The + # CENTRE ore patches and the ENEMY local patches live at the + # top-level `ore_patches:` block, which does NOT honour + # `spawn_point` — both north and south patches materialise on + # every seed (only the one on the agent's active latitude is + # within harv reach; the other is harmless flavor). Per + # CLAUDE.md the persistent far-edge sentinel is duplicated + # across both enemy spawn groups. Highest bar (4500) on the + # same 3600-tick clock as easy/medium — the intended forward + # placement still reaches ~5100 cash by T≈30 (the patch + # depletion ceiling), but a memorised "place at (60, 20)" + # opening fails because the centre patch is on y=14 or y=26 + # per seed and the proc footprint at (60,20) blockades + # nothing useful at the wrong y. + hard: + description: > + Your Construction Yard, Power Plant, and idle Ore Truck + spawn in the NORTH or SOUTH half of the map depending on + the seed; the mirrored enemy base (passive turtle bot) + mirrors your latitude. ONE rich ore patch sits on YOUR + spawn latitude (north patch on a north spawn; south patch + on a south spawn); a second mirror patch exists on the + OTHER latitude but is too far from your harv to be useful. + Build an Ore Refinery, place it ADJACENT to your latitude's + centre patch — `place_building` auto-spawns a harvester on + the patch — and order harvest there. economy_value ≥ 4500 + before tick 3600. Stalling, defensive-only play, and + own-base placement all lose on the clock. + starting_cash: 2000 + overrides: + actors: + # spawn_point 0 — NORTH agent base (y=14). Mirrored + # enemy base on the north latitude. Persistent sentinel + # duplicated for this spawn group. + - {type: fact, owner: agent, position: [15, 14], spawn_point: 0} + - {type: powr, owner: agent, position: [15, 12], spawn_point: 0} + - {type: harv, owner: agent, position: [18, 14], spawn_point: 0} + - {type: fact, owner: enemy, position: [113, 14], spawn_point: 0} + - {type: powr, owner: enemy, position: [113, 12], spawn_point: 0} + - {type: proc, owner: enemy, position: [108, 14], spawn_point: 0} + - {type: harv, owner: enemy, position: [110, 14], spawn_point: 0} + - {type: fact, owner: enemy, position: [124, 14], spawn_point: 0} + # spawn_point 1 — SOUTH agent base (y=26). Mirror. + - {type: fact, owner: agent, position: [15, 26], spawn_point: 1} + - {type: powr, owner: agent, position: [15, 28], spawn_point: 1} + - {type: harv, owner: agent, position: [18, 26], spawn_point: 1} + - {type: fact, owner: enemy, position: [113, 26], spawn_point: 1} + - {type: powr, owner: enemy, position: [113, 28], spawn_point: 1} + - {type: proc, owner: enemy, position: [108, 26], spawn_point: 1} + - {type: harv, owner: enemy, position: [110, 26], spawn_point: 1} + - {type: fact, owner: enemy, position: [124, 26], spawn_point: 1} + ore_patches: + # Both centre patches materialise on every seed (ore_patches + # do not honour spawn_point); only the one on the agent's + # active latitude is reachable. + - {x: 64, y: 14, amount: 12000, radius: 4} + - {x: 64, y: 26, amount: 12000, radius: 4} + # Enemy local patches, both latitudes, same justification. + - {x: 105, y: 14, amount: 3000, radius: 2} + - {x: 105, y: 26, amount: 3000, radius: 2} + win_condition: + all_of: + - has_building: proc + - economy_value_gte: 4500 + - within_ticks: 3600 + fail_condition: + any_of: + - after_ticks: 3601 + max_turns: 40 diff --git a/openra_bench/scenarios/schema.py b/openra_bench/scenarios/schema.py index 83cfbdf1ac4cb9f599bca475746d73de0a931cd7..6793ff91f8b1240242ea88daf4ff48996f62c310 100644 --- a/openra_bench/scenarios/schema.py +++ b/openra_bench/scenarios/schema.py @@ -261,6 +261,22 @@ class ScenarioPack(BaseModel): ge=0, description="Pack-wide economy budget; a level may override it.", ) + # Pack-wide top-level engine extras (RE-APPLIED — was reverted by an + # agent and the regression cost a full debug cycle). These live next + # to `base:` so a contributor can declare one set of `ore_patches:` / + # `water_cells:` / `scheduled_events:` once for the whole pack + # instead of restating them in every level's `overrides:`. A level + # may still override by restating the field inside `overrides:` + # (compile() falls back to the pack-level value only when the merged + # level value is empty). Without this declaration pydantic silently + # DROPS the top-level key, so `compile()` sees an empty list and the + # engine seeds zero ore / zero water cells / fires no events for + # easy + medium tiers (hard tier worked only because it re-declared + # the field inside its `overrides:` block). + ore_patches: list[dict[str, Any]] = Field(default_factory=list) + water_cells: list[list[int]] = Field(default_factory=list) + water_rect: list[int] | None = None + scheduled_events: list[dict[str, Any]] = Field(default_factory=list) levels: dict[LevelName, Level] # Optional named configurations. When present, the eval runs ONE # cell per config (pack:config_name) instead of the 3 raw levels — @@ -299,19 +315,19 @@ class ScenarioPack(BaseModel): # `_scenario_to_tmp_yaml` can reattach it to the engine YAML. # ScenarioDefinition ignores the field (extra='ignore') so # without this step the events would be silently dropped. - sched_events = list(merged.get("scheduled_events") or []) - # Resource-wave: lift `ore_patches:` for the same reason — - # ScenarioDefinition strips it silently. Without this lift the - # patches never reach the temp YAML and the engine seeds zero - # ore on the terrain. - ore_patches = list(merged.get("ore_patches") or []) - # Naval-MVP: lift `water_cells:` and `water_rect:` overlay - # blocks so `_scenario_to_tmp_yaml` can forward them to the - # engine. ScenarioDefinition strips them silently otherwise. + # Lift pack-wide → level-override; fall back to pack-level when + # a level didn't restate the field. (Was reverted by an agent; + # re-applied with the same pattern that was originally tested.) + sched_events = list( + merged.get("scheduled_events") or self.scheduled_events or [] + ) + ore_patches = list(merged.get("ore_patches") or self.ore_patches or []) water_cells = [ - list(c) for c in (merged.get("water_cells") or []) + list(c) for c in (merged.get("water_cells") or self.water_cells or []) ] water_rect = merged.get("water_rect") + if water_rect is None: + water_rect = self.water_rect return CompiledLevel( pack_id=self.meta.id, level=level, diff --git a/scripts/COLLECT_EVAL_DATA.md b/scripts/COLLECT_EVAL_DATA.md new file mode 100644 index 0000000000000000000000000000000000000000..6965bf77f1803e867812d0f2363e10c01089812a --- /dev/null +++ b/scripts/COLLECT_EVAL_DATA.md @@ -0,0 +1,189 @@ +# `collect_eval_data.py` — Phase 4 paper-collection driver + +Orchestrates per-cell audit-format eval runs across the Together AI +model roster, producing one JSONL (+ minimap PNG dir) per +`(model, pack, level, seed, fog_mode)` cell. Designed so a multi-hour, +multi-model collection can crash-resume losslessly and so every byte +needed for downstream paper analysis is captured at source — full +observations (including the `_raw` engine dict and the spatial +tensor), the literal HTTP request/response, engine warnings, per-turn +signals, and a `terminal:` block with outcome / wall-clock / token +totals. + +## Data layout + +``` +/ + _invocation.json # the CLI args + cost estimate + _summary.json # written on completion + .logs/ # per-cell stdout+stderr + ______seed__.log + ______seed__.stats.json + __/ # one dir per (timestamp, model) + ____seed__.jsonl + ____seed__/ + turn_001.png + turn_002.png + ... +``` + +The JSONL is one line per turn; each line has these fields (full +schema in `openra_bench/full_playback.py`): + +| field | description | +| ----------------- | ----------------------------------------------------------- | +| `turn` | int, 1-based turn index | +| `tick` | int, engine game tick at end of turn | +| `interrupt` | str or null — engine signal name that fired this turn | +| `obs` | dict, full `RustObsAdapter.render_state()` (includes `_raw` and `spatial`) | +| `briefing` | str, exactly the text the model received | +| `system_prompt` | str, first turn only; subsequent turns = null | +| `model_request` | dict, literal `{url, body}` posted to the provider | +| `model_response` | dict, literal `{raw, text, tool_calls, reasoning, usage, finish_reason}` | +| `commands_issued` | list[str], `Command.repr()` per command parsed | +| `engine_warnings` | list[str], from the Rust env's `info["warnings"]` | +| `signals` | dict, primitive signal snapshot (cash, kills, explored%, …) | +| `minimap_png` | str or null, relative path to the per-turn PNG | +| `done` | bool, engine `done` flag | +| `terminal` | present ONLY on the final line; see below | + +The `terminal` block has: + +```json +{ + "outcome": "win|loss|draw", + "final_obs": { ... }, + "wall_clock_seconds": 12.345, + "total_tokens_in": 78901, + "total_tokens_out": 1234, + "manifest": { /* scoring/score metadata */ } +} +``` + +## Invocation + +```bash +python3 scripts/collect_eval_data.py \ + --models Qwen/Qwen3.5-9B,Qwen/Qwen3.6-Plus,google/gemma-4-31B-it,moonshotai/Kimi-K2.6 \ + --packs all \ + --levels easy,medium,hard \ + --seeds 1,2,3,4 \ + --fog-modes vision \ + --run-label paper-collection-v1 \ + --output-dir data/runs/paper-collection-v1 \ + --parallel-cells 4 \ + --resume +``` + +* `--packs` accepts `all`, a comma list, `@file.txt` (one pack id per + line), or a directory of `*.yaml`. +* `--fog-modes` accepts any subset of + `structured,structured-clear,vision,vision-clear,image,image-clear`. +* `--parallel-cells` controls subprocess concurrency. Each cell is an + isolated `python -m openra_bench.run_eval` invocation so a crash in + one cell never aborts the rest of the run. + +Required env var (set in your shell / `.env`): `TOGETHER_API_KEY`. + +## Cost estimates + +`--cost-estimate` prints per-model and total token / USD estimates +WITHOUT spawning any subprocesses. The estimator uses average turns +and tokens-per-turn from the May 2026 pilot runs +(`playback/pilot_perception/`, `playback/pilot_handoff/`): + +* `avg_turns_per_cell = 18` (max_turns is typically 36; mean is ~half) +* `avg_prompt_tokens_per_turn = 4500` (briefing + image + codex, + with the 16-turn sliding window) +* `avg_completion_tokens_per_turn = 250` (a tool call + brief + reasoning) + +Pricing snapshot (USD per 1M tokens, May 2026): + +| model | in / M | out / M | +| ---------------------- | ------ | ------- | +| Qwen/Qwen3.5-9B | $0.20 | $0.20 | +| Qwen/Qwen3.6-Plus | $0.50 | $1.50 | +| qwen/qwen3.6-flash | $0.18 | $0.18 | +| google/gemma-4-31B-it | $0.25 | $0.25 | +| moonshotai/Kimi-K2.6 | $0.60 | $2.50 | + +Update `_TOGETHER_PRICES` at the top of `scripts/collect_eval_data.py` +when Together's published rates change. + +### Common run profiles (cost estimates) + +| profile | cells / model | total cells | est USD (4 models, 1 fog) | +| ----------------- | ------------- | ----------- | ------------------------- | +| **smoke**: 10 packs × 1 level × 1 seed × 1 fog | 10 | 40 | ~$1 | +| **mini**: 50 packs × 3 levels × 1 seed × 1 fog | 150 | 600 | ~$30 | +| **medium**: 50 packs × 3 levels × 4 seeds × 1 fog | 600 | 2400 | ~$120 | +| **full**: 200 packs × 3 levels × 4 seeds × 1 fog | 2400| 9600 | ~$480 | +| **perception sweep**: full × 6 fog modes | 14400| 57600| ~$2900 | + +Always run `--cost-estimate` before the real thing. + +## Resume / re-run a failed cell + +`--resume` scans the output dir and skips any cell whose JSONL is +complete (its last line has a `terminal:` field). Cells that crashed +mid-run leave a `.jsonl.partial` behind for forensics; the +final `.jsonl` is NOT created, so resume correctly retries them. + +To force a re-run of one cell, delete its `.jsonl` (and the +sibling PNG dir if you want a fresh image series). The next +`--resume` invocation will re-spawn it. + +For diagnostics, every cell's stdout+stderr is captured to +`/.logs/.log`. Tail one to see exactly what +`python -m openra_bench.run_eval` printed. + +## Loading the data for paper analysis + +```python +import json +from pathlib import Path + +def iter_cells(run_dir): + """Yield (path, lines) per cell — lines are list[dict].""" + for p in sorted(Path(run_dir).glob("**/*.jsonl")): + if p.name.startswith("_") or p.name.endswith(".partial"): + continue + with open(p) as fh: + lines = [json.loads(l) for l in fh if l.strip()] + yield p, lines + +for path, lines in iter_cells("data/runs/paper-collection-v1"): + term = lines[-1].get("terminal", {}) + outcome = term.get("outcome", "?") + n_turns = len(lines) + cost = ( + term.get("total_tokens_in", 0) / 1e6 * 0.5 + + term.get("total_tokens_out", 0) / 1e6 * 1.5 + ) + print(f"{path.stem:<60} {outcome:<5} turns={n_turns:>3} ~${cost:.3f}") +``` + +The viewer at `scripts/view_playback.py` transparently understands +the audit JSONL format alongside the legacy `seed/` dirs — point +it at `data/runs/` and it picks up both shapes. + +## Caveats / known limits + +* `--repeats > 1` currently shares the JSONL stem; for paper-grade + collection keep it at 1 (one cell == one deterministic seed). The + cell-level reliability metric (pass^k) belongs in a separate sweep + with distinct `--seeds`. +* `--full-playback` runs ALONGSIDE the legacy `Playback` — pass both + `--playback` and `--full-playback` to `run_eval` if you want the + human-readable viewer files AND the audit JSONL. The collector + script only emits the audit format (the playback dir is what the + viewer reads natively). +* Engine warnings reflect the `info["warnings"]` list emitted by the + Rust env at each step; the bench does NOT attach a model-level + warning channel (the model is judged only by its commands). +* `model_response.raw` carries the entire provider response JSON, + which on Together's side includes a `usage` block; downstream + paper analysis should pull token counts from there rather than + trusting any aggregate, because the per-turn breakdown is the + authoritative source. diff --git a/scripts/collect_eval_data.py b/scripts/collect_eval_data.py new file mode 100755 index 0000000000000000000000000000000000000000..25082070632bcc7d7cc51c74a49730cb34071858 --- /dev/null +++ b/scripts/collect_eval_data.py @@ -0,0 +1,537 @@ +#!/usr/bin/env python3 +"""Phase 4 paper-collection driver: spawn one subprocess per +(model, pack, level, seed, fog_mode) cell so a full Together-roster +eval produces complete, untruncated, audit-ready JSONL + PNG data. + +Each cell runs as an isolated `python -m openra_bench.run_eval` +invocation pinned to a SINGLE pack / level / seed / fog-mode, with the +audit recorder (`--full-playback `) writing one JSONL per cell: + + /__/____seed__.jsonl + /__/____seed__/turn_.png + +Behaviours: + +* `--resume`: skips a cell whose JSONL already has a `terminal:` line + (`openra_bench.full_playback.is_complete_cell`). A partial / crashed + cell is correctly retried. +* `--parallel-cells N`: up to N cell subprocesses run concurrently. + One crashing cell does not abort the run. +* `--dry-run`: prints the cell list + estimated token / USD cost + without spawning anything. +* `--cost-estimate`: same as dry-run for the cost lines only. +* Per-cell logs land in `/.logs/.log` so + post-hoc you can diagnose any cell that exited non-zero. + +Defaults are tuned for the Together AI roster the playback dirs use +(`Qwen/Qwen3.5-9B`, `Qwen/Qwen3.6-Plus`, `qwen/qwen3.6-flash`, +`google/gemma-4-31B-it`, `moonshotai/Kimi-K2.6`); override via flags. + +NOT a runner of the Together API directly: this script orchestrates +the existing `openra_bench.run_eval` CLI, which already speaks the +Together-compatible (OpenAI Chat Completions) wire format. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import shlex +import shutil +import subprocess +import sys +import time +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path + +# Allow running from a checkout without `pip install -e .`. +HERE = Path(__file__).resolve().parent +REPO = HERE.parent +if str(REPO) not in sys.path: + sys.path.insert(0, str(REPO)) + +from openra_bench.full_playback import cell_stem, is_complete_cell # noqa: E402 +from openra_bench.scenarios.loader import PACKS_DIR # noqa: E402 + +# ── Together AI pricing (USD per 1M tokens) ─────────────────────────── +# Snapshot as of Phase 4 prep (May 2026). Used by --cost-estimate. The +# values are conservative upper bounds — if Together's published price +# is lower today the estimate will overshoot, which is the right side +# to err on for budget approval. Update inline as prices change. +# +# Source: https://together.ai/pricing (look up each model) +_TOGETHER_PRICES: dict[str, tuple[float, float]] = { + # model_id_lower -> (in_per_M, out_per_M) + "qwen/qwen3.5-9b": (0.20, 0.20), + "qwen/qwen3.6-plus": (0.50, 1.50), + "qwen/qwen3.6-flash": (0.18, 0.18), + "google/gemma-4-31b-it": (0.25, 0.25), + "moonshotai/kimi-k2.6": (0.60, 2.50), +} +_DEFAULT_PRICE = (0.50, 1.50) # safe fallback if a model isn't listed + +# ── Token-per-turn priors (from playback/pilot_* historical runs) ───── +# Used by --cost-estimate. These are AVERAGES over the existing pilot +# runs (perception + handoff pilots, May 2026). Real numbers will vary +# per cell; the estimate is order-of-magnitude. +_AVG_TURNS_PER_CELL = 18 # mean across pilots; max_turns is 36 typically +_AVG_PROMPT_TOK_PER_TURN = 4500 # ~prompt+image+codex, sliding window +_AVG_COMPLETION_TOK_PER_TURN = 250 # tool call + brief reasoning + +# Default Together roster (from the existing playback dirs). +_DEFAULT_MODELS = [ + "Qwen/Qwen3.5-9B", + "Qwen/Qwen3.6-Plus", + "qwen/qwen3.6-flash", + "google/gemma-4-31B-it", + "moonshotai/Kimi-K2.6", +] + + +def _safe_model(m: str) -> str: + return re.sub(r"[^A-Za-z0-9._-]+", "_", m) + + +def _list_all_packs() -> list[str]: + """Every active (non-quarantine, non-template) pack id under + `openra_bench/scenarios/packs/`. Sorted for stable ordering.""" + out: list[str] = [] + for p in sorted(PACKS_DIR.glob("*.yaml")): + if p.name.startswith(("_", "TEMPLATE")): + continue + # Pack id == filename stem (matches load_pack convention). + out.append(p.stem) + return out + + +def _resolve_packs(spec: str | None) -> list[str]: + """Resolve `--packs`: + * `all` → every active pack + * `@FILE` → one pack id per line + * `a,b,c` → comma-separated list + * `path/to/dir` → every *.yaml in that dir (stem == id) + * None/empty → `all` + """ + if not spec or spec.lower() == "all": + return _list_all_packs() + if spec.startswith("@"): + ids = [] + for line in Path(spec[1:]).read_text().splitlines(): + line = line.strip() + if line and not line.startswith("#"): + ids.append(line) + return ids + p = Path(spec) + if p.is_dir(): + return sorted(x.stem for x in p.glob("*.yaml")) + return [s.strip() for s in spec.split(",") if s.strip()] + + +def _price_for(model: str) -> tuple[float, float]: + return _TOGETHER_PRICES.get(model.lower().strip(), _DEFAULT_PRICE) + + +def _estimate_cost(cells_per_model: dict[str, int]) -> dict: + """Per-model token + USD estimate from `cells_per_model`. Uses the + pilot-historical priors at module top; conservative on the high + side. Returns a printable structured dict.""" + rows = [] + total_usd = 0.0 + total_tok_in = 0 + total_tok_out = 0 + for model, n_cells in sorted(cells_per_model.items()): + in_per_m, out_per_m = _price_for(model) + tok_in = n_cells * _AVG_TURNS_PER_CELL * _AVG_PROMPT_TOK_PER_TURN + tok_out = n_cells * _AVG_TURNS_PER_CELL * _AVG_COMPLETION_TOK_PER_TURN + usd = (tok_in / 1_000_000) * in_per_m + (tok_out / 1_000_000) * out_per_m + rows.append( + { + "model": model, + "cells": n_cells, + "est_tokens_in": tok_in, + "est_tokens_out": tok_out, + "in_per_M": in_per_m, + "out_per_M": out_per_m, + "est_usd": round(usd, 2), + } + ) + total_usd += usd + total_tok_in += tok_in + total_tok_out += tok_out + return { + "per_model": rows, + "total_cells": sum(cells_per_model.values()), + "total_tokens_in": total_tok_in, + "total_tokens_out": total_tok_out, + "total_usd": round(total_usd, 2), + "assumptions": { + "avg_turns_per_cell": _AVG_TURNS_PER_CELL, + "avg_prompt_tokens_per_turn": _AVG_PROMPT_TOK_PER_TURN, + "avg_completion_tokens_per_turn": _AVG_COMPLETION_TOK_PER_TURN, + "source": "pilot pilot_perception + pilot_handoff means, May 2026", + }, + } + + +# ── Plan + run ──────────────────────────────────────────────────────── + + +def build_plan(args) -> list[dict]: + """Expand the args into the flat list of (model, pack, level, seed, + fog_mode, repeat) cells. Each item carries the JSONL path it will + write to so `--resume` can scan and skip.""" + models = [m.strip() for m in args.models.split(",") if m.strip()] + packs = _resolve_packs(args.packs) + levels = [s.strip() for s in args.levels.split(",") if s.strip()] + seeds = [int(s) for s in args.seeds.split(",") if s.strip()] + fogs = [s.strip() for s in args.fog_modes.split(",") if s.strip()] + rng_reps = range(max(1, int(args.repeats))) + + out: list[dict] = [] + out_root = Path(args.output_dir) + for model in models: + model_dir = out_root / f"{args.timestamp}__{_safe_model(model)}" + for pack in packs: + for level in levels: + for seed in seeds: + for fog in fogs: + for rep in rng_reps: + stem = cell_stem(pack, level, seed, fog) + if rep > 0: + stem = f"{stem}__rep{rep}" + jsonl = model_dir / f"{stem}.jsonl" + out.append( + { + "model": model, + "pack": pack, + "level": level, + "seed": seed, + "fog_mode": fog, + "repeat": rep, + "model_dir": str(model_dir), + "jsonl_path": str(jsonl), + "cell_id": ( + f"{_safe_model(model)}__{stem}" + ), + } + ) + return out + + +def filter_resume(plan: list[dict]) -> tuple[list[dict], list[dict]]: + """Split `plan` into (todo, done) on resume — `done` is anything + whose JSONL exists AND ends with a `terminal:` line.""" + todo: list[dict] = [] + done: list[dict] = [] + for c in plan: + if is_complete_cell(c["jsonl_path"]): + done.append(c) + else: + todo.append(c) + return todo, done + + +def _pack_path_for(pack_id: str) -> Path: + return PACKS_DIR / f"{pack_id}.yaml" + + +def _run_cell(cell: dict, args, python_bin: str) -> dict: + """Spawn one `python -m openra_bench.run_eval` for a single cell. + Returns a result dict with rc / log_path / jsonl_path.""" + model_dir = Path(cell["model_dir"]) + model_dir.mkdir(parents=True, exist_ok=True) + log_dir = Path(args.output_dir) / ".logs" + log_dir.mkdir(parents=True, exist_ok=True) + log_path = log_dir / f"{cell['cell_id']}.log" + + # The audit dir is the per-model timestamp dir; the inner JSONL stem + # is derived deterministically by FullPlayback from pack/level/seed/ + # fog. We pre-create the parent so concurrent cells don't race on it. + cmd = [ + python_bin, + "-m", + "openra_bench.run_eval", + "--provider", + args.provider, + "--model", + cell["model"], + "--packs", + str(_pack_path_for(cell["pack"])), + "--levels", + cell["level"], + "--seeds", + str(cell["seed"]), + "--fog-mode", + cell["fog_mode"], + "--full-playback", + str(model_dir), + # Always emit a per-cell stats file under .logs so the report has + # something even when --no-stats is fine to ignore. + "--out", + str(log_dir / f"{cell['cell_id']}.stats.json"), + # Empty seeds list arg quirk; we pin exactly one seed. + ] + # Resume / repeat: a per-cell `--repeats N` would multiply within + # the subprocess; we keep that to 1 (default) since the OUTER loop + # owns the repeat index — each repeat is a distinct subprocess so + # the JSONL stems differ (rep0 / repN). Future: thread the repeat + # index into the FullPlayback stem (the planner already encodes it + # in `cell_id`; the inner JSONL would still be `…__seedN__fog.jsonl` + # which a rep>0 cell would collide on — so we run rep0 only via + # this script for now and document repeats as a separate sweep). + + env = dict(os.environ) + # The CostMeter inside run_eval uses ProviderConfig.price_in/out_per_m + # which we don't pass here; cost capture is best-effort and the + # FullPlayback `terminal.total_tokens_*` is the authoritative token + # count for downstream costing. + + started = time.time() + with open(log_path, "w") as fh: + fh.write(f"# cmd: {shlex.join(cmd)}\n") + fh.write(f"# started: {time.strftime('%Y-%m-%d %H:%M:%S')}\n\n") + fh.flush() + try: + rc = subprocess.call( + cmd, stdout=fh, stderr=subprocess.STDOUT, env=env + ) + except Exception as e: # noqa: BLE001 + fh.write(f"\n# spawn failed: {type(e).__name__}: {e}\n") + rc = -1 + return { + "cell_id": cell["cell_id"], + "rc": rc, + "elapsed_s": round(time.time() - started, 2), + "log_path": str(log_path), + "jsonl_path": cell["jsonl_path"], + "complete": is_complete_cell(cell["jsonl_path"]), + } + + +def main(argv: list[str]) -> int: + ap = argparse.ArgumentParser( + prog="collect_eval_data.py", + description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + ap.add_argument( + "--models", + default=",".join(_DEFAULT_MODELS), + help="comma-separated Together model ids " + f"(default: {','.join(_DEFAULT_MODELS)})", + ) + ap.add_argument( + "--packs", + default="all", + help="`all` | comma-separated pack ids | @file_with_one_per_line", + ) + ap.add_argument("--levels", default="easy,medium,hard") + ap.add_argument("--seeds", default="1,2,3,4") + ap.add_argument( + "--fog-modes", + default="vision", + help="comma-separated subset of " + "structured,structured-clear,vision,vision-clear,image,image-clear", + ) + ap.add_argument( + "--repeats", + type=int, + default=1, + help="run each cell N times (rep0 + repN suffix). >1 currently " + "shares the JSONL stem — keep at 1 for paper-grade collection.", + ) + ap.add_argument( + "--run-label", + default="paper-collection", + help="logical name for this run (becomes a path segment under " + "--output-dir's parent if you want; informational here)", + ) + ap.add_argument( + "--output-dir", + default=None, + help="root for the audit dirs " + "(default: data/runs/)", + ) + ap.add_argument( + "--parallel-cells", + type=int, + default=1, + help="how many cell subprocesses to run at once", + ) + ap.add_argument( + "--provider", + default="together", + help="provider name forwarded to run_eval (default: together)", + ) + ap.add_argument( + "--resume", + action="store_true", + help="skip cells whose JSONL already ends with a terminal line", + ) + ap.add_argument( + "--dry-run", + action="store_true", + help="print the planned cell list + cost estimate; spawn nothing", + ) + ap.add_argument( + "--cost-estimate", + action="store_true", + help="print the cost estimate and exit (no plan listing)", + ) + ap.add_argument( + "--python", + default=sys.executable, + help=f"python binary used to launch run_eval (default: {sys.executable})", + ) + ap.add_argument( + "--timestamp", + default=None, + help="override the timestamp segment of the per-model dir " + "(default: now UTC, format YYYYMMDD-HHMMSS)", + ) + a = ap.parse_args(argv[1:]) + + a.timestamp = a.timestamp or time.strftime("%Y%m%d-%H%M%S", time.gmtime()) + if not a.output_dir: + a.output_dir = str(REPO / "data" / "runs" / a.run_label) + + plan = build_plan(a) + if not plan: + print("collect_eval_data: empty plan — nothing to do", file=sys.stderr) + return 2 + + # Cell counts per model for the cost estimator. + cells_per_model: dict[str, int] = {} + for c in plan: + cells_per_model[c["model"]] = cells_per_model.get(c["model"], 0) + 1 + cost = _estimate_cost(cells_per_model) + + if a.cost_estimate or a.dry_run: + print("== cost estimate ==") + for row in cost["per_model"]: + print( + f" {row['model']:<32} cells={row['cells']:>5} " + f"tok_in={row['est_tokens_in']:>10,} " + f"tok_out={row['est_tokens_out']:>9,} " + f"@ ${row['in_per_M']}/${row['out_per_M']} per M " + f"≈ ${row['est_usd']:>8.2f}" + ) + print( + f" TOTAL: cells={cost['total_cells']} " + f"tok_in={cost['total_tokens_in']:,} " + f"tok_out={cost['total_tokens_out']:,} " + f"≈ ${cost['total_usd']:.2f}" + ) + print(f" assumptions: {cost['assumptions']}") + if a.cost_estimate and not a.dry_run: + return 0 + + todo = plan + done: list[dict] = [] + if a.resume: + todo, done = filter_resume(plan) + print( + f"resume: {len(done)}/{len(plan)} cells already complete; " + f"{len(todo)} to run", + file=sys.stderr, + ) + + if a.dry_run: + print(f"\n== plan ({len(todo)} cells, {a.parallel_cells} parallel) ==") + # Truncate listing to a sane head; full plan goes to a sidecar. + for c in todo[:50]: + print( + f" {c['model']:<30} {c['pack']:<40} {c['level']:<6} " + f"seed={c['seed']} fog={c['fog_mode']:<18} -> " + f"{c['jsonl_path']}" + ) + if len(todo) > 50: + print(f" ... ({len(todo) - 50} more)") + # Sidecar plan dump for the reviewer. + sidecar = Path(a.output_dir) / "_dry_run_plan.json" + sidecar.parent.mkdir(parents=True, exist_ok=True) + sidecar.write_text( + json.dumps( + {"plan": todo, "cost": cost, "args": vars(a)}, + indent=2, + default=str, + ) + ) + print(f"\nwrote full plan: {sidecar}") + return 0 + + # ── Live run ────────────────────────────────────────────────────── + out_root = Path(a.output_dir) + out_root.mkdir(parents=True, exist_ok=True) + (out_root / ".logs").mkdir(parents=True, exist_ok=True) + # Manifest the invocation so post-hoc you can reproduce. + (out_root / "_invocation.json").write_text( + json.dumps( + { + "args": vars(a), + "cost_estimate": cost, + "planned": len(plan), + "todo": len(todo), + "resumed": len(done), + "started": time.strftime("%Y-%m-%d %H:%M:%SZ", time.gmtime()), + }, + indent=2, + default=str, + ) + ) + + results: list[dict] = [] + fail = 0 + completed = 0 + started = time.time() + if a.parallel_cells <= 1: + for c in todo: + r = _run_cell(c, a, a.python) + results.append(r) + completed += 1 if r["complete"] else 0 + fail += 0 if r["rc"] == 0 else 1 + print( + f"[{len(results)}/{len(todo)}] {c['cell_id']} " + f"rc={r['rc']} t={r['elapsed_s']}s " + f"complete={r['complete']}", + flush=True, + ) + else: + with ThreadPoolExecutor(max_workers=a.parallel_cells) as ex: + futs = {ex.submit(_run_cell, c, a, a.python): c for c in todo} + for fu in as_completed(futs): + r = fu.result() + results.append(r) + completed += 1 if r["complete"] else 0 + fail += 0 if r["rc"] == 0 else 1 + print( + f"[{len(results)}/{len(todo)}] {r['cell_id']} " + f"rc={r['rc']} t={r['elapsed_s']}s " + f"complete={r['complete']}", + flush=True, + ) + + elapsed = round(time.time() - started, 1) + summary = { + "ran": len(results), + "complete": completed, + "failed": fail, + "resumed": len(done), + "total_planned": len(plan), + "elapsed_seconds": elapsed, + "results": results, + } + (out_root / "_summary.json").write_text( + json.dumps(summary, indent=2, default=str) + ) + print( + f"\ndone in {elapsed}s: {completed}/{len(todo)} complete, " + f"{fail} failed; see {out_root}/_summary.json" + ) + return 0 if fail == 0 else 1 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/tests/test_apc_transport_end_to_end.py b/tests/test_apc_transport_end_to_end.py new file mode 100644 index 0000000000000000000000000000000000000000..10e25f255230c905db7e9009b7f0fdb86a14b046 --- /dev/null +++ b/tests/test_apc_transport_end_to_end.py @@ -0,0 +1,153 @@ +"""End-to-end guardrail: APC transport (Command.enter_transport → +APC drives → Command.unload) with infantry alive on the far side. + +Pins the full Python-side loop: + 1. An e1 issues `enter_transport(apc)` and walks adjacent → boards + (passenger is removed from the world and stashed in transport + cargo). + 2. The APC moves ~30 cells east via `move_units`. + 3. `unload(apc)` ejects the passenger onto an adjacent passable + cell; the infantry is alive and observable in the visible-units + list at the new (east) location. + +The Rust side has direct unit tests for each leg; this is the +bench-side mirror that proves Python `Command.enter_transport` / +`Command.move_units` / `Command.unload` actually drive the loop +through the env boundary. +""" + +from __future__ import annotations + +import tempfile +from pathlib import Path + +import pytest +import yaml + + +def _scenario_path(scenario: dict) -> str: + fd = tempfile.NamedTemporaryFile( + "w", suffix="_apc_transport.yaml", delete=False + ) + yaml.safe_dump(scenario, fd, sort_keys=False) + fd.close() + return fd.name + + +def test_apc_transport_loads_drives_and_unloads_infantry_alive(): + pytest.importorskip("openra_train") + pytest.importorskip("openra_rl_training") + from openra_train import Command + from openra_rl_training.training.rust_env_pool import RustEnvPool + + from openra_bench.rust_adapter import RustObsAdapter + + # APC at (20, 20), one passenger riflemen one cell south, and a + # far enemy actor so the world has a valid opponent footprint. + actors = [ + {"type": "apc", "owner": "agent", "position": [20, 20]}, + {"type": "e1", "owner": "agent", "position": [21, 20]}, + {"type": "e1", "owner": "enemy", "position": [90, 90]}, + ] + scenario = { + "name": "apc-transport-test", + "description": "engine guardrail: APC enter_transport/move/unload", + "base_map": "rush-hour-arena", + "starting_cash": 0, + "spawn_mcvs": False, + "agent": {"faction": "allies", "cash": 0}, + "enemy": {"faction": "soviet", "cash": 0}, + "tools": ["observe", "move_units", "enter_transport", "unload"], + "planning": True, + "termination": {"max_ticks": 20000}, + "actors": actors, + } + path = _scenario_path(scenario) + + pool = RustEnvPool(size=1, scenario_path=path) + env = pool.acquire() + try: + ad = RustObsAdapter() + ad.observe(env.reset(seed=1)) + rs0 = ad.render_state() + own0 = rs0.get("units_summary", []) or [] + apc_id = None + e1_id = None + for u in own0: + t = str(u.get("type", "")).lower() + if t == "apc" and apc_id is None: + apc_id = str(u["id"]) + elif t == "e1" and e1_id is None: + e1_id = str(u["id"]) + assert apc_id is not None and e1_id is not None, ( + f"need both APC and e1 in initial units_summary; got {own0}" + ) + + # 1. enter_transport: e1 boards the APC. + ad.observe(env.step([Command.enter_transport([e1_id], apc_id)])[0]) + boarded = False + for _ in range(80): + obs, _r, done, _i = env.step([Command.observe()]) + ad.observe(obs) + cur = ad.render_state().get("units_summary", []) or [] + # The passenger is removed from the world (stashed as + # cargo) when it boards. Detect by the e1 disappearing. + still_there = any(str(u["id"]) == e1_id for u in cur) + if not still_there: + boarded = True + break + if done: + break + assert boarded, ( + f"e1 ({e1_id}) must board the APC ({apc_id}) within the step " + f"budget; current units={ad.render_state().get('units_summary')}" + ) + + # 2. APC drives ~30 cells east via move_units. + dest_x, dest_y = 55, 20 + ad.observe(env.step([Command.move_units([apc_id], dest_x, dest_y)])[0]) + arrived = False + for _ in range(200): + obs, _r, done, _i = env.step([Command.observe()]) + ad.observe(obs) + cur = ad.render_state().get("units_summary", []) or [] + apc = next((u for u in cur if str(u["id"]) == apc_id), None) + if apc is None: + break + if abs(int(apc["cell_x"]) - dest_x) <= 2 and abs(int(apc["cell_y"]) - dest_y) <= 2: + arrived = True + break + if done: + break + assert arrived, ( + f"APC must move within 2 cells of ({dest_x},{dest_y}) within the " + f"step budget; final units={ad.render_state().get('units_summary')}" + ) + + # 3. unload: passenger ejected onto a passable cell adjacent + # to the APC; visible in units_summary again. + ad.observe(env.step([Command.unload([apc_id])])[0]) + unloaded = False + for _ in range(20): + obs, _r, done, _i = env.step([Command.observe()]) + ad.observe(obs) + cur = ad.render_state().get("units_summary", []) or [] + e1s = [u for u in cur if str(u.get("type", "")).lower() == "e1"] + for u in e1s: + # Must be near the APC's destination, not near the + # initial (20,20) position — proves the APC actually + # carried the unit east. + cx, cy = int(u["cell_x"]), int(u["cell_y"]) + if abs(cx - dest_x) <= 3 and abs(cy - dest_y) <= 3: + unloaded = True + break + if unloaded or done: + break + assert unloaded, ( + f"unload must eject the e1 near the APC's new location " + f"({dest_x},{dest_y}); final units={ad.render_state().get('units_summary')}" + ) + finally: + pool.release(env) + pool.shutdown() + Path(path).unlink(missing_ok=True) diff --git a/tests/test_data_collection.py b/tests/test_data_collection.py new file mode 100644 index 0000000000000000000000000000000000000000..9bf1cebba7ea5001c1daea24a066a54d272dc511 --- /dev/null +++ b/tests/test_data_collection.py @@ -0,0 +1,207 @@ +"""Phase 4 data-collection plumbing: audit-format JSONL + the +`scripts/collect_eval_data.py` driver. + +Three smoke tests that don't need a Together API key: + +* `test_dry_run_plan_is_valid` — the planner expands the right cells + and a `--dry-run` invocation produces a parseable plan sidecar. +* `test_full_playback_writes_audit_jsonl` — a live 1-cell run with a + fake-model controller produces a valid JSONL whose last line carries + a `terminal:` block, plus the per-turn minimap PNGs. +* `test_resume_skips_completed_cell` — `is_complete_cell` correctly + detects a finished cell so `--resume` skips it. + +All tests use the bench's scripted controllers; no network / no +provider call. +""" + +from __future__ import annotations + +import json +import subprocess +import sys +from pathlib import Path + +import pytest + +pytest.importorskip("openra_train", reason="Rust env wheel not installed") +pytest.importorskip("openra_rl_training", reason="Rust env wheel not installed") + +from openra_bench.eval_core import run_level +from openra_bench.full_playback import ( + FullPlayback, + cell_stem, + is_complete_cell, +) +from openra_bench.scenarios import load_pack +from openra_bench.scenarios.loader import PACKS_DIR, compile_level + +# Smallest deterministic pack the rest of the suite already exercises. +PACK = PACKS_DIR / "perception-frontier-reading.yaml" + + +# ── Planner / CLI dry-run ──────────────────────────────────────────── + + +def test_dry_run_plan_is_valid(tmp_path: Path): + """The collector script's --dry-run produces a sidecar plan JSON + that lists exactly the cells implied by the inputs, without + spawning any subprocess.""" + out = tmp_path / "runs" / "smoke" + cmd = [ + sys.executable, + str(Path(__file__).resolve().parents[1] / "scripts" / "collect_eval_data.py"), + "--models", + "Qwen/Qwen3.5-9B,google/gemma-4-31B-it", + "--packs", + "perception-frontier-reading,action-multiunit-coordination", + "--levels", + "easy,medium", + "--seeds", + "1,2", + "--fog-modes", + "vision,structured", + "--run-label", + "smoke", + "--output-dir", + str(out), + "--dry-run", + ] + rc = subprocess.call(cmd) + assert rc == 0, "dry-run must exit 0" + sidecar = out / "_dry_run_plan.json" + assert sidecar.exists(), "dry-run sidecar plan should be written" + data = json.loads(sidecar.read_text()) + # 2 models * 2 packs * 2 levels * 2 seeds * 2 fogs = 32 cells + assert len(data["plan"]) == 32, ( + f"expected 32 cells in plan, got {len(data['plan'])}" + ) + # Cost lines must be present and finite. + assert data["cost"]["total_usd"] > 0 + assert data["cost"]["total_cells"] == 32 + + +# ── Audit JSONL shape ──────────────────────────────────────────────── + + +class _ScriptedAgent: + """Stand-in ModelAgent: exposes `history` (so legacy playback works) + AND the audit-capture attrs (`audit_capture`, `last_briefing`, + `last_request`, `last_response`, `system_prompt`) so a FullPlayback + line is exercised end-to-end without a network call.""" + + def __init__(self): + self.history = [{"role": "system", "content": "SYS"}] + self.stats = {"turns": 0, "tool_calls": 0, "empty_replies": 0} + # FullPlayback contract. + self.audit_capture = False + self.system_prompt = "FAKE SYSTEM PROMPT" + self.last_briefing = "" + self.last_request: dict | None = None + self.last_response: dict | None = None + + def agent_fn(self, render_state, Command): + self.stats["turns"] += 1 + self.last_briefing = ( + f"turn {self.stats['turns']}: tick={render_state.get('game_tick')}" + ) + self.last_request = { + "url": "https://fake/api", + "body": {"model": "fake", "messages": []}, + } + self.last_response = { + "raw": {"id": f"fake-{self.stats['turns']}"}, + "text": "stalling", + "tool_calls": [{"name": "observe", "arguments": {}}], + "reasoning": "", + "usage": {"prompt_tokens": 100, "completion_tokens": 5}, + "finish_reason": "stop", + } + return [Command.observe()] + + +def test_full_playback_writes_audit_jsonl(tmp_path: Path): + c = compile_level(load_pack(PACK), "easy") + agent = _ScriptedAgent() + fp = FullPlayback( + tmp_path, pack_id=c.pack_id, level="easy", seed=1, fog_mode="vision" + ) + res = run_level(c, agent.agent_fn, seed=1, playback=None, full_playback=fp) + + jsonl = tmp_path / f"{cell_stem(c.pack_id, 'easy', 1, 'vision')}.jsonl" + assert jsonl.exists(), "audit JSONL must land at the canonical stem" + lines = [json.loads(x) for x in jsonl.read_text().splitlines() if x] + assert lines, "audit JSONL must have at least one line" + # One line per model turn (no separate terminal frame — the terminal + # block merges into the LAST turn line). + assert len(lines) == res.turns, ( + f"expected {res.turns} lines, got {len(lines)}" + ) + first = lines[0] + # System prompt only on turn 1. + assert first["system_prompt"] == "FAKE SYSTEM PROMPT" + if len(lines) > 1: + assert lines[1]["system_prompt"] is None + # Full obs must be present (not truncated). + assert isinstance(first["obs"], dict) and "minimap" in first["obs"] + # Briefing + model request/response captured. + assert first["briefing"].startswith("turn 1") + assert first["model_request"]["url"] == "https://fake/api" + assert first["model_response"]["text"] == "stalling" + # Engine warnings carried through (may be empty list — that's fine). + assert isinstance(first["engine_warnings"], list) + # Commands captured as repr strings (audit format). + assert first["commands_issued"] and "Observe" in first["commands_issued"][0] + # Last line has the terminal block with totals. + term = lines[-1]["terminal"] + assert term["outcome"] in {"win", "loss", "draw"} + assert term["wall_clock_seconds"] >= 0 + # Token totals are simply summed from the per-turn responses (5 out per + # turn in the fake → res.turns * 5). + assert term["total_tokens_out"] == res.turns * 5 + + # is_complete_cell sees the terminal marker. + assert is_complete_cell(jsonl) is True + + +# ── Resume detection ───────────────────────────────────────────────── + + +def test_resume_skips_completed_cell(tmp_path: Path): + """`is_complete_cell` returns True for a JSONL whose last line has a + `terminal:` block, False otherwise. This is what `--resume` relies + on to skip cells without re-spawning their subprocess.""" + # A finished cell. + done = tmp_path / "done.jsonl" + done.write_text( + json.dumps({"turn": 1, "tick": 93, "obs": {}}) + "\n" + + json.dumps( + { + "turn": 2, + "tick": 183, + "obs": {}, + "terminal": { + "outcome": "win", + "wall_clock_seconds": 1.0, + "total_tokens_in": 0, + "total_tokens_out": 0, + }, + } + ) + + "\n" + ) + assert is_complete_cell(done) is True + + # An interrupted cell: no terminal on the last line. + half = tmp_path / "half.jsonl" + half.write_text( + json.dumps({"turn": 1, "tick": 93, "obs": {}}) + "\n" + + json.dumps({"turn": 2, "tick": 183, "obs": {}}) + "\n" + ) + assert is_complete_cell(half) is False + + # An empty file / missing file. + assert is_complete_cell(tmp_path / "missing.jsonl") is False + empty = tmp_path / "empty.jsonl" + empty.write_text("") + assert is_complete_cell(empty) is False diff --git a/tests/test_engineer_capture.py b/tests/test_engineer_capture.py index 8dc184526131da73e4bbcd10c3425bda31d4e8f2..492a2b0896d5773109a390f167175607ff1339ca 100644 --- a/tests/test_engineer_capture.py +++ b/tests/test_engineer_capture.py @@ -49,7 +49,7 @@ _PACK_YAML = textwrap.dedent( termination: {max_ticks: 6000} actors: - {type: e6, owner: agent, position: [16, 20]} - - {type: proc, owner: enemy, position: [22, 20]} + - {type: proc, owner: enemy, position: [20, 20]} levels: easy: description: 'engineer walks ~6 cells east and captures the proc' @@ -107,18 +107,18 @@ def test_engineer_captures_enemy_proc_via_command_capture_actor(): ad.observe(env.reset(seed=1)) render0 = ad.render_state() - # Locate the engineer (own_units) and the enemy proc (enemy_buildings or + # Locate the engineer (units_summary) and the enemy proc (enemy_buildings or # enemy actors surfaced in the snapshot). - own_units = render0.get("own_units", []) or [] - e6s = [u for u in own_units if str(u.get("type", "")).lower() == "e6"] - assert len(e6s) == 1, f"expected exactly one e6 in own_units, got {own_units}" + units_summary = render0.get("units_summary", []) or [] + e6s = [u for u in units_summary if str(u.get("type", "")).lower() == "e6"] + assert len(e6s) == 1, f"expected exactly one e6 in units_summary, got {units_summary}" engineer_id = str(e6s[0]["id"]) # The enemy proc surfaces via `enemy_buildings` / actor enumeration. We # accept either presentation — locate it by type==proc and enemy ownership. enemy_buildings = ( - render0.get("enemy_buildings") - or render0.get("enemies", []) + render0.get("enemy_buildings_summary") + or render0.get("enemy_summary", []) or [] ) proc_id = None @@ -164,8 +164,8 @@ def test_engineer_captures_enemy_proc_via_command_capture_actor(): f"to the agent within the timeout (own_buildings={own_b_final})" ) - # Engineer must be consumed (no e6 left in own_units). - own_units_final = ad.render_state().get("own_units", []) or [] + # Engineer must be consumed (no e6 left in units_summary). + own_units_final = ad.render_state().get("units_summary", []) or [] live_e6 = [ u for u in own_units_final if str(u.get("type", "")).lower() == "e6" ] diff --git a/tests/test_infiltrate.py b/tests/test_infiltrate.py index 4e03ac7434366385b2e35d778eebe33979e9d9ce..a8a63503b678d508e6ffe59b7006830b0afdb111 100644 --- a/tests/test_infiltrate.py +++ b/tests/test_infiltrate.py @@ -162,7 +162,7 @@ def test_thief_steals_cash_via_infiltrate(): ) assert silo_id is not None, "enemy silo must be visible to thief at start" - cash_before = int(ad.render_state().get("economy", {}).get("cash", 0)) + cash_before = int(ad.render_state().get("cash", 0)) obs, _r, done, _i = env.step( [env.Command.infiltrate([str(thf_id)], str(silo_id))] @@ -179,7 +179,7 @@ def test_thief_steals_cash_via_infiltrate(): str(u.get("type", "")).lower() == "thf" for u in own_after ), f"thief must have been consumed ({own_after})" - cash_after = int(ad.render_state().get("economy", {}).get("cash", 0)) + cash_after = int(ad.render_state().get("cash", 0)) assert cash_after > cash_before, ( f"thief infiltration must increase agent cash " f"(before={cash_before}, after={cash_after})" diff --git a/tests/test_per_player_starting_cash.py b/tests/test_per_player_starting_cash.py new file mode 100644 index 0000000000000000000000000000000000000000..25453800134caf91234c76206690329a5299fd67 --- /dev/null +++ b/tests/test_per_player_starting_cash.py @@ -0,0 +1,150 @@ +"""Per-player starting-cash plumbing: end-to-end via `openra_train`. + +Pins the engine fix that the bench's `agent: {cash: N}` / +`enemy: {cash: M}` scenario-YAML knobs are now honoured per-player at +world build time. Before the fix the engine plumbed a single +`starting_cash: int` into ALL player actors, so an +`agent: {cash: 0}` + `enemy: {cash: 1500}` scenario gave both sides +the SAME cash (0), which broke scenarios like +`spec-thief-steal-cash` (the thief had nothing to steal). + +Two sub-tests: + * `test_agent_and_enemy_cash_independent` — both per-player + overrides are present; the agent starts at 500 and the enemy at + 1500. + * `test_back_compat_no_per_player_cash` — when the scenario omits + `agent: {cash:}` and `enemy: {cash:}`, both slots inherit the + top-level `starting_cash:` (the pre-fix behaviour, which the + fix preserves as the default). +""" + +from __future__ import annotations + +import tempfile +from pathlib import Path + +import pytest +import yaml + + +def _scenario( + actors: list[dict], + *, + starting_cash: int = 0, + agent_cash: int | None = None, + enemy_cash: int | None = None, +) -> dict: + agent = {"faction": "allies"} + if agent_cash is not None: + agent["cash"] = agent_cash + enemy = {"faction": "soviet"} + if enemy_cash is not None: + enemy["cash"] = enemy_cash + return { + "name": "per-player-cash-test", + "description": "per-player starting-cash plumbing", + "base_map": "rush-hour-arena", + "starting_cash": starting_cash, + "spawn_mcvs": False, + "agent": agent, + "enemy": enemy, + "tools": ["observe"], + "planning": True, + "termination": {"max_ticks": 600}, + "actors": actors, + } + + +def _scenario_path(scenario: dict) -> str: + fd = tempfile.NamedTemporaryFile("w", suffix="_per_player_cash.yaml", delete=False) + yaml.safe_dump(scenario, fd, sort_keys=False) + fd.close() + return fd.name + + +def test_agent_and_enemy_cash_independent(): + pytest.importorskip("openra_train") + from openra_rl_training.training.rust_env_pool import RustEnvPool + + from openra_bench.rust_adapter import RustObsAdapter + + # Minimal scene: one agent rifleman + one enemy rifleman so the + # world builds cleanly. The cash assertions don't depend on + # combat — we just snapshot tick 0. + actors = [ + {"type": "e1", "owner": "agent", "position": [20, 20]}, + {"type": "e1", "owner": "enemy", "position": [60, 60]}, + ] + path = _scenario_path( + _scenario(actors, starting_cash=0, agent_cash=500, enemy_cash=1500) + ) + pool = RustEnvPool(size=1, scenario_path=path) + env = pool.acquire() + try: + ad = RustObsAdapter() + obs = env.reset(seed=1) + ad.observe(obs) + + # The agent's own cash is surfaced as the top-level `cash` + # field by RustObsAdapter.render_state() (mirrors the + # `cash` key in the raw obs). + agent_cash = int(ad.render_state().get("cash", -1)) + assert agent_cash == 500, ( + f"agent slot must honour `agent: {{cash: 500}}` " + f"(got {agent_cash})" + ) + + # Cross-check enemy cash via the engine's `player_cash` + # accessor exposed on the env handle. Available since + # the per-player-cash fix shipped. + # RustEnvHandle wraps the native `openra_train.OpenRAEnv`; + # the per-player accessors live on the inner object. + inner = env._env + enemy_pid = inner.enemy_player_id + enemy_cash = inner.player_cash(enemy_pid) + assert enemy_cash == 1500, ( + f"enemy slot must honour `enemy: {{cash: 1500}}` " + f"(got {enemy_cash})" + ) + finally: + pool.release(env) + pool.shutdown() + Path(path).unlink(missing_ok=True) + + +def test_back_compat_no_per_player_cash(): + pytest.importorskip("openra_train") + from openra_rl_training.training.rust_env_pool import RustEnvPool + + from openra_bench.rust_adapter import RustObsAdapter + + actors = [ + {"type": "e1", "owner": "agent", "position": [20, 20]}, + {"type": "e1", "owner": "enemy", "position": [60, 60]}, + ] + # Neither agent.cash nor enemy.cash set ⇒ both slots inherit + # the top-level 750 (back-compat with every existing pack). + path = _scenario_path(_scenario(actors, starting_cash=750)) + pool = RustEnvPool(size=1, scenario_path=path) + env = pool.acquire() + try: + ad = RustObsAdapter() + ad.observe(env.reset(seed=1)) + agent_cash = int(ad.render_state().get("cash", -1)) + assert agent_cash == 750, ( + f"agent slot must inherit lobby `starting_cash: 750` " + f"when `agent.cash` is omitted (got {agent_cash})" + ) + # RustEnvHandle wraps the native `openra_train.OpenRAEnv`; + # the per-player accessors live on the inner object. + inner = env._env + enemy_pid = inner.enemy_player_id + enemy_cash = inner.player_cash(enemy_pid) + assert enemy_cash == 750, ( + f"enemy slot must inherit lobby `starting_cash: 750` " + f"when `enemy.cash` is omitted (got {enemy_cash})" + ) + finally: + pool.release(env) + pool.shutdown() + Path(path).unlink(missing_ok=True) diff --git a/tests/test_proc_auto_spawn_python.py b/tests/test_proc_auto_spawn_python.py new file mode 100644 index 0000000000000000000000000000000000000000..a83d4579571f8c5d0ee39095e40f3c2984024250 --- /dev/null +++ b/tests/test_proc_auto_spawn_python.py @@ -0,0 +1,160 @@ +"""End-to-end guardrail: a 2nd `proc` auto-spawns its `harv` adjacent +to the NEW proc (not piled on top of the lowest-id proc), and the +fresh harv picks the path-shortest refinery as its delivery target. + +Historical footgun (closed by the matching engine commit): + * `order_place_building` called `spawn_unit("harv", owner)`, which + routed through `find_spawn_location` — that helper sorts + production-building candidates by `(!is_primary, id)`, so the + auto-harv always materialised next to the LOWEST-id proc, never + the new one. A 2nd refinery placed far from the 1st gained no + throughput from its own auto-harv. + * `find_refinery` returned the first (lowest-id) `proc`, so every + harv deposited at the closest by ID, not by path-distance. Adding + a refinery near a contested patch never paid off. + +Mirror of `OpenRA-Rust/openra-sim/tests/test_proc_auto_spawn_at_new_proc.rs`, +exercised here via the Python `OpenRAEnv` boundary so the bench-side +adapter is pinned too. +""" + +from __future__ import annotations + +import tempfile +from pathlib import Path + +import pytest +import yaml + + +def _scenario(actors, *, agent_cash: int = 5000) -> dict: + return { + "name": "proc-auto-spawn-test", + "description": "engine guardrail: 2nd proc auto-harv lands near the NEW proc", + "base_map": "rush-hour-arena", + "starting_cash": agent_cash, + "spawn_mcvs": False, + "agent": {"faction": "allies", "cash": agent_cash}, + "enemy": {"faction": "soviet", "cash": 0}, + "tools": ["observe", "build", "place_building"], + "planning": True, + "termination": {"max_ticks": 12000}, + "actors": actors, + } + + +def _scenario_path(scenario: dict) -> str: + fd = tempfile.NamedTemporaryFile( + "w", suffix="_proc_spawn.yaml", delete=False + ) + yaml.safe_dump(scenario, fd, sort_keys=False) + fd.close() + return fd.name + + +def test_second_proc_autospawns_harv_at_the_new_proc_via_python_env(): + pytest.importorskip("openra_train") + pytest.importorskip("openra_rl_training") + from openra_train import Command + from openra_rl_training.training.rust_env_pool import RustEnvPool + + from openra_bench.rust_adapter import RustObsAdapter + + # Pre-place a small base on the WEST: fact + powr + 1st proc. + # Enemy gets a single far rifleman so the world has a valid + # opponent footprint (no draw-on-elim). + actors = [ + {"type": "fact", "owner": "agent", "position": [10, 10]}, + {"type": "powr", "owner": "agent", "position": [14, 10]}, + {"type": "powr", "owner": "agent", "position": [16, 10]}, + {"type": "proc", "owner": "agent", "position": [10, 14]}, + {"type": "e1", "owner": "enemy", "position": [90, 90]}, + ] + path = _scenario_path(_scenario(actors, agent_cash=8000)) + + pool = RustEnvPool(size=1, scenario_path=path) + env = pool.acquire() + try: + ad = RustObsAdapter() + ad.observe(env.reset(seed=1)) + + # Pre-place harv id snapshot (so we can identify the new one + # by id-order later). + own_units0 = ad.render_state().get("units_summary", []) or [] + pre_harv_ids = { + int(u["id"]) + for u in own_units0 + if str(u.get("type", "")).lower() == "harv" + } + + # Queue the 2nd proc. cost ≈ 1400; one Build call enqueues + # one item, then we step until done. + ad.observe(env.step([Command.build("proc")])[0]) + + # Step until the proc completes (production tick uses + # ~3 frames per process_frame; loop until the raw observation + # surfaces the proc item with `done=True`, or budget + # exhausted). render_state's `production` is collapsed to + # item strings; we reach for the raw obs via the env handle. + done_proc = False + for _ in range(120): + obs, _r, _d, _i = env.step([Command.observe()]) + ad.observe(obs) + raw_prod = obs.get("production", []) or [] + if any( + isinstance(p, dict) + and str(p.get("item", "")).lower() == "proc" + and bool(p.get("done", False)) + for p in raw_prod + ): + done_proc = True + break + assert done_proc, ( + "2nd proc must be completed in the production queue before " + "place_building can fire" + ) + + # Place the 2nd proc FAR EAST. + east_x, east_y = 70, 14 + ad.observe(env.step([Command.place_building("proc", east_x, east_y)])[0]) + # Step one more to fire the SpawnUnit frame-end task. + ad.observe(env.step([Command.observe()])[0]) + + rs = ad.render_state() + own_units = rs.get("units_summary", []) or [] + harvs = [ + u + for u in own_units + if str(u.get("type", "")).lower() == "harv" + ] + new_harvs = [u for u in harvs if int(u["id"]) not in pre_harv_ids] + own_b = rs.get("own_buildings", []) or [] + procs = [b for b in own_b if str(b.get("type", "")).lower() == "proc"] + assert len(procs) >= 2, ( + f"expected ≥2 procs after place_building; got own_buildings={own_b}" + ) + assert len(new_harvs) >= 1, ( + "placing a 2nd proc must auto-spawn a 2nd harv " + f"(pre={pre_harv_ids}, " + f"post_harvs={[(u['id'], u['cell_x'], u['cell_y']) for u in harvs]}, " + f"procs={procs})" + ) + new_harv = new_harvs[0] + hx, hy = int(new_harv["cell_x"]), int(new_harv["cell_y"]) + + # Chebyshev distance to the NEW (east) vs OLD (west) proc. + cheb_east = max(abs(hx - east_x), abs(hy - east_y)) + cheb_west = max(abs(hx - 10), abs(hy - 14)) + assert cheb_east <= 3, ( + f"new harv must spawn within 3 cells of the NEW (east) proc; " + f"harv at ({hx},{hy}), east proc at ({east_x},{east_y}), " + f"Chebyshev distance={cheb_east}" + ) + assert cheb_east < cheb_west, ( + f"new harv must be CLOSER to the new (east) proc than to the " + f"old (west) proc; east={cheb_east} west={cheb_west}" + ) + finally: + pool.release(env) + pool.shutdown() + Path(path).unlink(missing_ok=True) diff --git a/tests/test_resource_economy.py b/tests/test_resource_economy.py index ac27ca15ac27701d0dbb75f4dfa6ff2b9f256071..93fce89285282936b3659369fc7ccb3f2a497358 100644 --- a/tests/test_resource_economy.py +++ b/tests/test_resource_economy.py @@ -53,7 +53,11 @@ _PACK_YAML = textwrap.dedent( base_map: rush-hour-arena starting_cash: 2000 base: - agent: {faction: allies} + # `agent.cash` must be set explicitly — per-player cash plumbing + # (PlayerSetup.cash) now defaults to 0 and silently overrides the + # pack-level `starting_cash:`. Without this line the agent starts + # with $0 and the build('proc') queue stalls indefinitely. + agent: {faction: allies, cash: 2000} enemy: {faction: soviet, cash: 0} tools: [observe, build, place_building, move_units, stop] spawn_mcvs: false @@ -146,11 +150,13 @@ def test_stall_policy_yields_zero_income(): try: obs = env.reset(seed=1) starting_cash = int(obs["economy"]["cash"]) + last_obs = obs for _ in range(15): # 15 decision turns × ~90 ticks ≈ 1350 ticks - _ = env.step([openra_train.Command.observe()]) - # observation() after step - final_obs = env.last_observation() - final_cash = int(final_obs["economy"]["cash"]) + last_obs, _r, _done, _i = env.step( + [openra_train.Command.observe()] + ) + # env.step() returns (obs, reward, done, info) — no last_observation(). + final_cash = int(last_obs["economy"]["cash"]) assert final_cash <= starting_cash, ( f"stall must not grow cash (no proc ⇒ harv idle); " f"start={starting_cash} final={final_cash}" @@ -176,22 +182,26 @@ def test_intended_policy_grows_cash_via_harvest(): starting_total = starting_cash + int(obs["economy"].get("resources", 0)) # Turn 1: queue the refinery. - _ = env.step([openra_train.Command.build("proc")]) - # Spin a few turns to let production complete (proc is 1400 - # cost, build time ~ a few turns). - for _ in range(6): - _ = env.step([openra_train.Command.observe()]) - - # Turn N: place proc adjacent to the ore patch centre (24, 20). - # `place_building` does not enforce build-adjacency, so we drop - # it directly next to the ore. - _ = env.step([openra_train.Command.place_building("proc", 21, 20)]) - - # Spin out the remaining decision budget. + _o, _r, _d, _i = env.step([openra_train.Command.build("proc")]) + # Wait for production to complete, then retry place_building until + # it's accepted (premature place is a no-op + logs "PLACE BLOCKED"). + placed = False + for _ in range(25): + o, _r, _d, _i = env.step( + [openra_train.Command.place_building("proc", 21, 20)] + ) + if [b for b in (o.get("own_buildings") or []) + if b.get("type") == "proc"]: + placed = True + break + + assert placed, "proc never landed (production stalled or place_building refused)" + + # Spin out the remaining decision budget. Harv auto-routes the + # moment the proc is alive; one full cycle is ~12-15 turns. grew = False - for _ in range(20): - _ = env.step([openra_train.Command.observe()]) - o = env.last_observation() + for _ in range(30): + o, _r, _d, _i = env.step([openra_train.Command.observe()]) total = int(o["economy"]["cash"]) + int( o["economy"].get("resources", 0) ) diff --git a/tests/test_tanya_c4.py b/tests/test_tanya_c4.py index 04267afe7a9ee6b2a7ee129f8411da19e8dca181..204f549e8bd64a51489656b57023e8ee409d52a6 100644 --- a/tests/test_tanya_c4.py +++ b/tests/test_tanya_c4.py @@ -49,7 +49,7 @@ _PACK_YAML = textwrap.dedent( termination: {max_ticks: 6000} actors: - {type: tanya, owner: agent, position: [16, 20]} - - {type: proc, owner: enemy, position: [22, 20]} + - {type: proc, owner: enemy, position: [20, 20]} levels: easy: description: 'tanya walks ~6 cells east and C4s the proc' @@ -106,16 +106,16 @@ def test_tanya_c4_destroys_enemy_proc_via_command_c4_detonate(): ad.observe(env.reset(seed=1)) render0 = ad.render_state() - own_units = render0.get("own_units", []) or [] - tanyas = [u for u in own_units if str(u.get("type", "")).lower() == "tanya"] + units_summary = render0.get("units_summary", []) or [] + tanyas = [u for u in units_summary if str(u.get("type", "")).lower() == "tanya"] assert len(tanyas) == 1, ( - f"expected exactly one tanya in own_units, got {own_units}" + f"expected exactly one tanya in units_summary, got {units_summary}" ) tanya_id = str(tanyas[0]["id"]) enemy_buildings = ( - render0.get("enemy_buildings") - or render0.get("enemies", []) + render0.get("enemy_buildings_summary") + or render0.get("enemy_summary", []) or [] ) proc_id = None @@ -148,8 +148,8 @@ def test_tanya_c4_destroys_enemy_proc_via_command_c4_detonate(): ad.observe(obs) rs = ad.render_state() enemy_b = ( - rs.get("enemy_buildings") - or rs.get("enemies", []) + rs.get("enemy_buildings_summary") + or rs.get("enemy_summary", []) or [] ) still_proc = any( @@ -169,13 +169,13 @@ def test_tanya_c4_destroys_enemy_proc_via_command_c4_detonate(): ) # Tanya survives the detonation. - own_units_final = ad.render_state().get("own_units", []) or [] + own_units_final = ad.render_state().get("units_summary", []) or [] live_tanya = [ u for u in own_units_final if str(u.get("type", "")).lower() == "tanya" ] assert live_tanya, ( "tanya should survive the C4 detonation; she is missing from " - "own_units after the blast" + "units_summary after the blast" ) finally: pool.release(env)