# HITL Trace Implementation Validation

## Implementation Summary

The HITL (Human-In-The-Loop) checkpoint review steps are now recorded as separate steps in `trace.json` for later review, in addition to existing event emission in `run_events.jsonl`.

## Key Changes

### File Modified
- `src/data_agent_baseline/langgraph_agent/graph.py`

### New Function Added
- `_record_hitl_step()`: Records HITL checkpoint decisions as trace steps

### Integration Points
HITL steps are recorded at 4 decision points in `_maybe_handle_planner_checkpoint()`:

1. **Approve Path**
   - Decision: User approves the planner's original plan
   - Step recorded with original_plan = revised_plan

2. **Cancel Path**
   - Decision: User cancels the task
   - Step recorded with human_instruction included
   - Includes cancellation reason in message

3. **Revise Path**
   - Decision: User submits steering instruction for replanning
   - Replanning occurs via planner_node()
   - Step recorded with both original_plan and revised_plan
   - Includes delta comparison for changes

4. **Timeout Auto-Approve Path**
   - Decision: Timeout elapsed, plan auto-approved
   - Step recorded with timeout_seconds and default_action_on_timeout

## Trace.json Structure

Each HITL step in `trace.json["steps"]` has this structure:

```json
{
  "step_index": <number>,
  "action": "planner_review",
  "phase": "planner_review",
  "thought": "Human-in-the-loop checkpoint review (checkpoint_id=...)",
  "action_input": {
    "checkpoint_id": "cp_...",
    "decision": "approve|cancel|revise|timeout_auto_approve"
  },
  "raw_response": "(human decision)",
  "observation": {
    "ok": true,
    "tool": "planner_review",
    "content": {
      "checkpoint_id": "cp_...",
      "decision": "...",
      "message": "...",
      "original_plan": { ... },
      "revised_plan": { ... },
      "human_instruction": "...",
      "timeout_seconds": 60,
      "default_action_on_timeout": "continue"
    }
  },
  "ok": true
}
```

## Usage Example

When viewing trace.json after a guided run with HITL:

```bash
cd /data3/dataFAIR/kdd-dev/public/artifacts/runs/
jq '.steps[-1]' <run_id>/task_<id>/trace.json  # View last HITL step
```

This shows:
- The checkpoint ID
- The human decision (approve, cancel, or revise)
- The original plan that was presented
- The revised plan (if applicable)
- Human steering instructions (if applicable)
- Timeout configuration

## Backward Compatibility

- No changes to existing trace structure
- HITL steps are appended to steps array
- Existing HITL event emission in run_events.jsonl unchanged
- Non-HITL runs are unaffected

## Verification

To verify implementation:

```bash
# 1. Run a guided task with HITL enabled
python main.py --mode guided --task-id task_145 ...

# 2. Check trace.json for HITL step
cat artifacts/runs/<run_id>/task_145/trace.json | jq '.steps[] | select(.action == "planner_review")'

# 3. Verify run_events.jsonl still has HITL events
cat artifacts/runs/<run_id>/run_events.jsonl | grep checkpoint_reached
```

## Benefits

- ✅ HITL decisions persist in trace.json for audit/review
- ✅ Full checkpoint lifecycle visible (creation, decision, resolution)
- ✅ Original and revised plans captured together
- ✅ Human steering instructions preserved
- ✅ Compatible with existing visualization tools
- ✅ Available for long-term historical analysis