IMPLEMENTATION_ROADMAP.md

# QueueOps OpenEnv Implementation Roadmap

This roadmap is the execution reference for building the real-world queueing environment in this repository.

Constraints locked in:
- Keep existing directory structure unchanged.
- Treat `cloud_queue_env/` as the project root.
- Use HF token provider flow in `inference.py`.
- Follow OpenEnv compliance strictly: typed models, `step()/reset()/state()`, valid `openenv.yaml`.
- Provide deterministic graders with partial scoring in `[0, 1]`.
- Deliver at least 3 tasks (more optional).

---

## V1 - Hackathon-Ready Submission

Goal: submit a valid, real-world OpenEnv benchmark with 3 deterministic graded tasks and reproducible inference outputs.

### Phase 1 - Core Simulator Foundation
Sub-goals:
1. Replace echo logic with queue-operations simulation core.
2. Add deterministic RNG with explicit seed handling.
3. Implement proper episode boundaries (`horizon`, terminal conditions).
4. Keep strict OpenEnv contract for `reset()`, `step()`, and `state`.

Definition of done:
- Environment no longer behaves as dummy echo.
- Same seed + same action trace => identical trajectory.
- Episode always terminates predictably.

### Phase 2 - Task System (Easy/Medium/Hard)
Sub-goals:
1. Add task selection (`task_id`) and per-task config.
2. Implement Task A (single queue, admission control).
3. Implement Task B (multi-server, priority routing).
4. Implement Task C (two-stage queue network, dynamic scaling/cost).

Definition of done:
- All 3 tasks run end-to-end from `reset()` to terminal state.
- Difficulty progression is visible from A -> B -> C.

### Phase 3 - Deterministic Graders + Partial Scoring
Sub-goals:
1. Implement per-task grader formulas from master spec.
2. Keep each grader output bounded in `[0, 1]`.
3. Handle invalid/NaN/infinite values safely and deterministically.
4. Aggregate final benchmark score as mean of task scores.

Definition of done:
- Repeated runs on same seeds produce same grader outputs.
- Partial scoring is meaningful (not binary pass/fail only).

### Phase 4 - Reward Shaping and Safety Penalties
Sub-goals:
1. Add dense reward components: wait, throughput, SLA, cost, fairness, safety.
2. Add penalties for invalid actions and exploit patterns.
3. Bound reward scale across tasks.
4. Expose reward components in `info` for debugging.

Definition of done:
- Reward moves through trajectory, not only at the end.
- Unsafe or degenerate behavior is penalized.

### Phase 5 - Inference Protocol Compliance
Sub-goals:
1. Update `inference.py` to run all required tasks with fixed seeds.
2. Keep OpenAI client usage while authenticating with HF token flow.
3. Emit strict `[START]`, `[STEP]`, `[END]` line format.
4. Print per-task and final aggregate scores.

Definition of done:
- Script executes benchmark sweep reproducibly.
- Output format matches hackathon requirements.

### Phase 6 - Packaging, Validation, Documentation
Sub-goals:
1. Validate `openenv.yaml` metadata and app wiring.
2. Confirm Docker build/run success.
3. Update README with task definitions, action/observation spaces, reward/grader equations, baseline results.
4. Verify deployment readiness for HF Space.

Definition of done:
- OpenEnv validation passes.
- Container starts and serves correctly.
- README is submission-ready.

### V1 Submission Gate
All must be true:
1. 3 tasks implemented and deterministic.
2. Graders return valid partial scores in `[0, 1]`.
3. Inference script reports reproducible benchmark outputs.
4. OpenEnv spec compliance confirmed.
5. Docker and README requirements satisfied.

---

## V2 - Quality and Robustness Upgrade

Goal: improve benchmark reliability, score stability, and anti-exploit behavior after initial submission.

### Phase 1 - Determinism Hardening
Sub-goals:
1. Split RNG streams (arrivals/service/abandonment/shocks).
2. Add trace replay support for debugging.
3. Extend `info` with deterministic audit fields.

### Phase 2 - Difficulty Calibration
Sub-goals:
1. Tune parameters for cleaner A/B/C separation.
2. Improve level interpolation behavior.
3. Add stronger guards against reject-all or noop exploitation.

### Phase 3 - Reporting and Confidence
Sub-goals:
1. Add standardized per-seed report table.
2. Add mean/std summaries over seed sets.
3. Flag unstable metrics and grader edge cases.

### V2 Exit Criteria
1. Lower run-to-run variance on fixed seed sets.
2. Clearer task difficulty progression.
3. Better fairness and exploit resistance.

---

## V3 - Extended Benchmark Pack (Optional)

Goal: increase novelty and long-term benchmark value with optional extra tasks.

### Phase 1 - Task D (Non-stationary Load)
Sub-goals:
1. Add shift-based and bursty arrivals.
2. Grade robustness under changing demand.

### Phase 2 - Task E (Partial Observability)
Sub-goals:
1. Add delayed/noisy metrics.
2. Grade safe decisions under uncertainty.

### Phase 3 - Public Benchmark Packaging
Sub-goals:
1. Publish official seed suites.
2. Add benchmark profiles: quick / standard / full.
3. Provide reference baseline outputs.

### V3 Exit Criteria
1. 4-5 total tasks available.
2. Broader real-world coverage.
3. Stronger benchmark differentiation.

---

## Execution Order

Recommended order:
1. Complete V1 fully and submit.
2. Continue with V2 for quality hardening.
3. Do V3 only if timeline allows.

Immediate next implementation step:
- Start V1 Phase 1 (models + simulator core + deterministic state transitions).