docs/overview/EXECUTIVE_VISUALIZATION_GUIDE.md

Executive Visualization - Usage Guide

This guide shows how to use the new executive-friendly visualization features for stakeholder demonstrations.

🎯 Overview

The visualization module transforms technical logging into business-friendly reports that highlight:

• Decision Reasoning: Why the agent chose a particular approach
• Execution Journey: Timeline of attempts and refinements
• Confidence Metrics: Quality indicators based on execution patterns
• Clear Outcomes: Success/failure with actionable insights

📋 Commands Available

1. Run Task with Executive Mode (Live Demo)

Use this for: Live demonstrations to executives

# Run with executive-friendly display (modern, clean output)
uv run dabench run-lang-task task_355 --config configs/react_baseline.azure.yaml --display-mode=executive

# Run with technical display (default - detailed logs)
uv run dabench run-lang-task task_355 --config configs/react_baseline.azure.yaml --display-mode=technical

What you see:

• Executive mode: Clean panels showing strategy, execution timeline, and results
• Technical mode: Original detailed logging (unchanged, default behavior)

2. View Executive Report (Post-Run Analysis)

Use this for: Analyzing completed tasks, preparing presentations

# View report for a specific task from a run
uv run dabench view-exec-report task_355 20260601T075638Z

# Or use full path
uv run dabench view-exec-report task_355 /data3/dataFAIR/kdd-dev/public/artifacts/runs/20260601T075638Z

What you see:

• 📊 Executive Summary with status and timing
• 🧠 Decision Reasoning (the "why" behind the approach)
• ⚙️ Execution Timeline (refinement cycles)
• 📦 Deliverables (output location)

3. Run Multiple Tasks with Executive Display

# Run multiple tasks for comprehensive demo
uv run dabench run-lang-task task_355 task_330 task_418 \
  --config configs/react_baseline.azure.yaml \
  --display-mode=executive

🎨 Visual Style

The executive mode uses:

• ✅ ❌ Status emojis for quick scanning
• 🔍 📋 ⚙️ Phase indicators
• 🟢 🟡 🟠 🔴 Confidence color coding
• Rounded panels with clear typography
• Business terminology (not technical jargon)

📊 Example Output

Executive Mode Output

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📊 Executive Summary: task_355
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

╭─────────── Overview ────────────╮
│ Status              ✅ Success   │
│ Completion Time     ⏱️  1m 41s   │
│ Refinement Cycles   🔄 5 attempts│
│ Confidence Level    🟠 Moderate  │
╰─────────────────────────────────╯

╭──────────── 🧠 Decision Reasoning ─────────────╮
│ 💡 Strategy: Find records where expense       │
│ description matches 'Water, Veggie tray and   │
│ supplies', then connect to member records to  │
│ retrieve first_name, last_name, and cost.     │
│                                                │
│ 📋 Target Output: first_name, last_name, cost │
│ 📊 Expected Result: single row(s)             │
╰────────────────────────────────────────────────╯

⚙️  Execution & Refinement Timeline
┌─────────┬──────────┬──────────────────────────────┐
│ Attempt │  Status  │ Insight / Action Taken       │
├─────────┼──────────┼──────────────────────────────┤
│   #1    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #2    │ ❌ Refine│ Search criteria too          │
│         │          │ restrictive - broadening     │
│   #3    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #4    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
│   #5    │ ❌ Refine│ No matching records found -  │
│         │          │ refining search criteria     │
└─────────┴──────────┴──────────────────────────────┘

╭────────── 📦 Deliverables ───────────╮
│ 📁 Results File    .../prediction.csv│
│ 📊 Output Columns  first_name,       │
│                    last_name, cost   │
╰──────────────────────────────────────╯

🔄 Migration Notes

Backward Compatibility

• Default behavior unchanged: All existing commands work exactly as before
• Opt-in only: Executive mode is only active when explicitly requested
• No breaking changes: Technical mode is still the default

When to Use Each Mode

Audience	Recommended Mode	Why
Software Engineers	technical	Full debugging details, logs, stack traces
Data Scientists	technical	Technical validation, performance metrics
Senior Executives	executive	Business outcomes, decision reasoning
Stakeholder Demos	executive	Clear narrative, visual appeal
Post-Run Analysis	Use view-exec-report	Polished presentation of results

🚀 Quick Start for Demo

For your executive demonstration:

# 1. Run a task with executive display
uv run dabench run-lang-task task_355 \
  --config configs/react_baseline.azure.yaml \
  --display-mode=executive

# 2. Or view a report for an already-completed run
uv run dabench view-exec-report task_355 20260601T075638Z

📁 Files Created

src/data_agent_baseline/visualization/
├── __init__.py              # Module exports
├── trace_parser.py          # Parse trace.json structure
├── executive_reporter.py    # Generate formatted reports
├── narratives.py            # Technical → Business language mapping
└── live_display.py          # Live demo display (future enhancement)

🔧 Technical Details

• trace.json parsing: Extracts decision reasoning, execution attempts, timing
• Narrative translation: Maps technical actions to business-friendly descriptions
• Rich formatting: Uses Rich library for beautiful terminal output
• No side effects: Read-only operations, doesn't modify any data