visualization/README.md

# AI Messaging System - Visualization Tool

An interactive Streamlit-based visualization and experimentation tool for the AI Messaging System v2. This tool enables non-technical users to generate, visualize, analyze, and improve personalized message campaigns with integrated A/B testing capabilities and comprehensive historical tracking.

## 🎯 Purpose

- Generate personalized messages with various configurations
- Run A/B tests with parallel processing and side-by-side comparison
- Visualize and analyze messages in real-time across all campaign stages
- Provide detailed feedback with message header/body tracking
- Track improvements and trends across historical experiments
- Cloud-native architecture with Snowflake integration
- Ready for deployment on HuggingFace Spaces

---

## 🏗️ Architecture Overview

### Core Philosophy: **In-Memory + Cloud Persistence**

The system operates on a hybrid architecture:
- **In-Memory Operations**: All active experiments run in `session_state` (no local files)
- **Cloud Persistence**: Data stored in Snowflake for long-term analytics
- **On-Demand Loading**: Historical data loaded from Snowflake when needed
- **One-Click Storage**: Results persisted with a single button click

### Why This Architecture?

1. **HuggingFace Ready**: No local file dependencies
2. **Fast Operations**: In-memory processing for real-time feedback
3. **Scalable**: Snowflake handles unlimited historical data
4. **Clean Separation**: Current experiments vs. historical data
5. **Versioned Configs**: Automatic configuration versioning in Snowflake

---

## 📁 Directory Structure

```
visualization/
├── app.py                         # Main entry point with authentication & brand selection
├── pages/                         # Multi-page Streamlit application
│   ├── 1_Campaign_Builder.py      # Campaign configuration & generation (with A/B testing)
│   ├── 2_Message_Viewer.py        # Message browsing & feedback (A/B aware)
│   ├── 4_Analytics.py             # Performance metrics for CURRENT experiment
│   └── 5_Historical_Analytics.py  # Historical experiments from Snowflake
├── utils/                         # Utility modules
│   ├── __init__.py
│   ├── auth.py                    # Authentication logic
│   ├── config_manager.py          # Configuration loading from Snowflake
│   ├── db_manager.py              # Snowflake database operations (NEW)
│   ├── experiment_runner.py       # Parallel experiment execution (NEW)
│   ├── session_feedback_manager.py # In-memory feedback management (NEW)
│   └── theme.py                   # Brand-specific theming
├── data/                          # Local data storage (configs cached here)
│   └── UI_users/                  # Pre-loaded user lists (100 users per brand)
│       ├── drumeo_users.csv
│       ├── pianote_users.csv
│       ├── guitareo_users.csv
│       └── singeo_users.csv
├── requirements.txt               # Python dependencies
├── README.md                      # This file
├── IMPLEMENTATION_COMPLETE.md     # Refactoring details & progress
└── ARCHITECTURE_REFACTOR_GUIDE.md # Technical refactoring guide
```

### Deprecated Files (No Longer Used)

These files are legacy and no longer part of the active codebase:
- ~~`utils/data_loader.py`~~ - Replaced by session_state loading
- ~~`utils/feedback_manager.py`~~ - Replaced by SessionFeedbackManager
- ~~`data/configs/`~~ - Configs now cached locally but loaded from Snowflake
- ~~`data/feedback/`~~ - Feedback now in session_state → Snowflake
- ~~`ai_messaging_system_v2/Data/ui_output/`~~ - No more file outputs

---

## 🗄️ Snowflake Database Schema

### Tables

**1. MESSAGING_SYSTEM_V2.UI.CONFIGS**
```sql
CONFIG_NAME      VARCHAR    -- Configuration identifier
CONFIG_FILE      VARIANT    -- JSON configuration
CONFIG_VERSION   INTEGER    -- Auto-incrementing version
BRAND            VARCHAR    -- Brand name (drumeo, pianote, etc.)
CREATED_AT       TIMESTAMP  -- Creation timestamp
```

**2. MESSAGING_SYSTEM_V2.UI.EXPERIMENT_METADATA**
```sql
EXPERIMENT_ID               VARCHAR    -- Unique experiment identifier
CONFIG_NAME                 VARCHAR    -- Configuration used
BRAND                       VARCHAR    -- Brand name
CAMPAIGN_NAME               VARCHAR    -- Campaign identifier
STAGE                       INTEGER    -- Stage number (1-11)
LLM_MODEL                   VARCHAR    -- Model used (gpt-4o-mini, gemini-2.5-flash-lite, etc.)
TOTAL_MESSAGES              INTEGER    -- Messages generated in this stage
TOTAL_USERS                 INTEGER    -- Unique users in this stage
PLATFORM                    VARCHAR    -- Platform (push, email, etc.)
PERSONALIZATION             BOOLEAN    -- Personalization enabled
INVOLVE_RECSYS              BOOLEAN    -- Recommendations enabled
RECSYS_CONTENTS             ARRAY      -- Recommendation types
SEGMENT_INFO                VARCHAR    -- Segment description
CAMPAIGN_INSTRUCTIONS       VARCHAR    -- Campaign-wide instructions
PER_MESSAGE_INSTRUCTIONS    VARCHAR    -- Stage-specific instructions
START_TIME                  TIMESTAMP  -- Experiment start time
END_TIME                    TIMESTAMP  -- Experiment end time (optional)
```

**3. MESSAGING_SYSTEM_V2.UI.FEEDBACKS**
```sql
EXPERIMENT_ID      VARCHAR    -- Links to EXPERIMENT_METADATA
USER_ID            INTEGER    -- User who received the message
STAGE              INTEGER    -- Stage number
FEEDBACK_TYPE      VARCHAR    -- 'reject' (only type currently)
REJECTION_REASON   VARCHAR    -- Reason category key
REJECTION_TEXT     VARCHAR    -- Custom text explanation
MESSAGE_HEADER     VARCHAR    -- Full message header
MESSAGE_BODY       VARCHAR    -- Full message body
CAMPAIGN_NAME      VARCHAR    -- Campaign identifier
BRAND              VARCHAR    -- Brand name
CONFIG_NAME        VARCHAR    -- Configuration used
TIMESTAMP          TIMESTAMP  -- Feedback submission time
```

---

## 🚀 Getting Started

### Prerequisites

- Python 3.9+
- Snowflake account and credentials
- OpenAI/Google AI API keys
- AI Messaging System v2 installed in parent directory
- Plotly for data visualization

### Installation

```bash
# Install dependencies
pip install -r requirements.txt

# Set up environment variables (.env file)
SNOWFLAKE_USER=your_user
SNOWFLAKE_PASSWORD=your_password
SNOWFLAKE_ACCOUNT=your_account
SNOWFLAKE_ROLE=your_role
SNOWFLAKE_DATABASE=MESSAGING_SYSTEM_V2
SNOWFLAKE_WAREHOUSE=your_warehouse
SNOWFLAKE_SCHEMA=UI

# LLM API keys
OPENAI_API_KEY=your_key
GOOGLE_API_KEY=your_key
```

### Running the Application

```bash
# From the visualization directory
cd visualization
streamlit run app.py
```

### First-Time Setup

1. Ensure `.env` file exists with Snowflake credentials
2. Verify user CSV files exist in `data/UI_users/` for each brand
3. Create Snowflake tables using schema above (or let system auto-create)
4. Upload initial configurations to Snowflake (optional - can create in UI)
5. Login with authorized email and access token

---

## 📊 Complete Data Flow

### **Session State Architecture**

All active data lives in Streamlit's `session_state`:

```python
# Single Experiment Mode
st.session_state.ui_log_data              # DataFrame: Generated messages
st.session_state.current_experiment_id    # String: Experiment identifier
st.session_state.current_experiment_metadata  # List[Dict]: Metadata per stage
st.session_state.current_feedbacks        # List[Dict]: Feedback records

# AB Testing Mode
st.session_state.ui_log_data_a            # DataFrame: Experiment A messages
st.session_state.ui_log_data_b            # DataFrame: Experiment B messages
st.session_state.experiment_a_id          # String: Experiment A identifier
st.session_state.experiment_b_id          # String: Experiment B identifier
st.session_state.experiment_a_metadata    # List[Dict]: A's metadata
st.session_state.experiment_b_metadata    # List[Dict]: B's metadata
st.session_state.feedbacks_a              # List[Dict]: A's feedback
st.session_state.feedbacks_b              # List[Dict]: B's feedback

# Configuration
st.session_state.campaign_config          # Dict: Single mode config
st.session_state.campaign_config_a        # Dict: AB mode config A
st.session_state.campaign_config_b        # Dict: AB mode config B
st.session_state.configs_cache            # Dict: Cached configs from Snowflake
```

### **End-to-End Workflow**

```
┌─────────────────────────────────────────────────────────────────┐
│                     1. CAMPAIGN BUILDER                         │
├─────────────────────────────────────────────────────────────────┤
│ • Load configs from Snowflake (cached in session_state)         │
│ • User selects/modifies configuration                           │
│ • Sample users from CSV files                                   │
│ • Generate messages (stored in session_state.ui_log_data)       │
│ • Track metadata (session_state.current_experiment_metadata)    │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│                      2. MESSAGE VIEWER                          │
├─────────────────────────────────────────────────────────────────┤
│ • Load messages from session_state.ui_log_data                  │
│ • Display in user-centric or stage-centric views               │
│ • User provides feedback (reject messages)                      │
│ • Feedback stored in session_state.current_feedbacks            │
│ • [BUTTON] Store Results to Snowflake                          │
│   → Writes metadata to EXPERIMENT_METADATA table               │
│   → Writes feedback to FEEDBACKS table                         │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│                        3. ANALYTICS                             │
├─────────────────────────────────────────────────────────────────┤
│ • Load current experiment from session_state                    │
│ • Calculate metrics using SessionFeedbackManager                │
│ • Show overall performance, stage analysis, rejection reasons   │
│ • Support AB testing side-by-side comparison                    │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│                   4. HISTORICAL ANALYTICS                       │
├─────────────────────────────────────────────────────────────────┤
│ • [BUTTON] Load Historical Data from Snowflake                  │
│ • Query EXPERIMENT_METADATA + FEEDBACKS tables                  │
│ • Calculate aggregate metrics across all experiments            │
│ • Show trends: rejection rates over time                        │
│ • Compare configurations: which performs best                   │
│ • Filter by date range                                          │
└─────────────────────────────────────────────────────────────────┘
```

---

## 📄 Pages Overview

### Page 0: Home & Authentication (app.py)

**Purpose**: Login, brand selection, config loading, and navigation hub

**Features**:
- Email and token-based authentication
- Brand selection (Drumeo, Pianote, Guitareo, Singeo)
- **Config loading from Snowflake on startup**
- Brand-specific theming applied throughout app
- Navigation guide and quick start instructions
- Current experiment status overview

**Technical Details**:
- Session-based authentication
- Brand selection persists via `st.session_state.selected_brand`
- Configs cached in `st.session_state.configs_cache` on first load
- Dynamic theming using `utils/theme.py`
- Loads environment variables from `.env` file

**Key Functions**:
```python
def load_configs_from_snowflake(brand):
    """Load all configs for brand from Snowflake, cache in session_state."""
    session = create_snowflake_session()
    config_manager = ConfigManager(session)
    configs = config_manager.load_configs_from_snowflake(brand)
    st.session_state.configs_cache = configs
    session.close()
```

---

### Page 1: Campaign Builder

**Purpose**: Create and run message generation campaigns with integrated A/B testing

**Architecture**: In-memory experiment execution with parallel AB testing

#### Key Features

**Configuration Management**:
- Loads configs from cached `session_state.configs_cache`
- Real-time config editing in UI
- Save configs to Snowflake with auto-versioning
- Quick save or save-as-new options

**A/B Testing Toggle**:
- Single Experiment Mode (default): One campaign
- A/B Testing Mode: Two parallel experiments for comparison

**User Sampling**:
- Random sampling from brand-specific CSV files
- 1-25 users selectable
- Same users used for both AB experiments (fair comparison)

**Parallel Execution**:
- Uses `ExperimentRunner.run_ab_test_parallel()`
- Threading for simultaneous A/B generation
- Separate Snowflake sessions per experiment (no conflicts)
- Console logging for thread progress
- Results stored directly in session_state

**Technical Details**:

**Single Mode Generation**:
```python
# ExperimentRunner handles all generation logic
runner = ExperimentRunner(brand=brand, system_config=config)

success, ui_log_data, metadata = runner.run_single_experiment(
    config=campaign_config,
    sampled_users_df=users_df,
    experiment_id=experiment_id,
    create_session_func=create_snowflake_session,
    progress_container=st.container()
)

# Store in session_state
st.session_state.ui_log_data = ui_log_data
st.session_state.current_experiment_metadata = metadata
st.session_state.current_experiment_id = experiment_id
```

**AB Mode Generation**:
```python
# Parallel execution in threads
results = runner.run_ab_test_parallel(
    config_a=campaign_config_a,
    config_b=campaign_config_b,
    sampled_users_df=users_df,
    experiment_a_id=experiment_a_id,
    experiment_b_id=experiment_b_id,
    create_session_func=create_snowflake_session
)

# Store both results
st.session_state.ui_log_data_a = results['a']['ui_log_data']
st.session_state.ui_log_data_b = results['b']['ui_log_data']
st.session_state.experiment_a_metadata = results['a']['metadata']
st.session_state.experiment_b_metadata = results['b']['metadata']
```

**Configuration Saving to Snowflake**:
```python
# Save with auto-versioning
session = create_snowflake_session()
db_manager = UIDatabaseManager(session)
success = db_manager.save_config(
    config_name=config_name,
    config_data=campaign_config,
    brand=brand
)
session.close()
```

**Thread-Safe Progress Handling**:
- Main thread: Full Streamlit UI updates
- Worker threads: Console logging only
- Dummy progress bars prevent Streamlit errors in threads

---

### Page 2: Message Viewer

**Purpose**: Browse, search, and evaluate generated messages with A/B testing awareness

**Architecture**: Loads from session_state, feedback stored in-memory

#### Key Features

**Automatic AB Detection**:
```python
def detect_ab_testing_mode():
    """Detect AB mode from session_state."""
    return (
        'ui_log_data_a' in st.session_state and
        'ui_log_data_b' in st.session_state and
        st.session_state.ui_log_data_a is not None and
        st.session_state.ui_log_data_b is not None
    )
```

**Message Loading**:
```python
# Single mode
def get_single_experiment_data():
    if 'ui_log_data' in st.session_state:
        return st.session_state.ui_log_data
    return None

# AB mode - loads both dataframes
messages_a_df = st.session_state.ui_log_data_a
messages_b_df = st.session_state.ui_log_data_b
```

**Feedback System**:
- In-memory storage using `SessionFeedbackManager`
- Rejection categories: Poor header, Poor body, Grammar, Emoji, Recommendation issues, Similar to previous, etc.
- Stores full message header and body with feedback
- Undo rejection capability

```python
# Add feedback
SessionFeedbackManager.add_feedback(
    experiment_id=experiment_id,
    user_id=user_id,
    stage=stage,
    feedback_type="reject",
    rejection_reason="poor_header",
    rejection_text="Too generic",
    message_header=header,
    message_body=body,
    campaign_name=campaign_name,
    brand=brand,
    config_name=config_name,
    feedback_list_key="current_feedbacks"  # or "feedbacks_a", "feedbacks_b"
)
```

**Store Results to Snowflake** (CRITICAL FEATURE):

Located after message viewing section. Button appears prominently.

```python
if st.button("💾 Store Results to Snowflake"):
    session = create_snowflake_session()
    db_manager = UIDatabaseManager(session)

    # Single mode
    if not ab_mode:
        # Store metadata
        for meta in st.session_state.current_experiment_metadata:
            db_manager.store_experiment_metadata(meta)

        # Store feedback
        for feedback in st.session_state.current_feedbacks:
            db_manager.store_feedback(feedback)

    # AB mode
    else:
        # Store both experiments
        for meta in st.session_state.experiment_a_metadata:
            db_manager.store_experiment_metadata(meta)
        for meta in st.session_state.experiment_b_metadata:
            db_manager.store_experiment_metadata(meta)

        for feedback in st.session_state.feedbacks_a:
            db_manager.store_feedback(feedback)
        for feedback in st.session_state.feedbacks_b:
            db_manager.store_feedback(feedback)

    session.close()
    st.success("✅ Results stored to Snowflake successfully!")
    st.balloons()
```

**View Modes**:
- User-Centric: All stages for each user
- Stage-Centric: All users for each stage
- Filters: Stage selection, keyword search, pagination

---

### Page 3: Analytics Dashboard

**Purpose**: Visualize performance metrics for CURRENT experiment only

**Architecture**: Loads from session_state, uses SessionFeedbackManager

#### Key Features

**Data Loading**:
```python
# Single mode
def get_single_experiment_data():
    return st.session_state.ui_log_data

# AB mode
detect_ab_testing_mode()  # Returns True if AB data exists
messages_a_df = st.session_state.ui_log_data_a
messages_b_df = st.session_state.ui_log_data_b
```

**Feedback Stats Calculation**:
```python
# Single mode
feedback_stats = SessionFeedbackManager.get_feedback_stats(
    experiment_id=experiment_id,
    total_messages=len(messages_df),
    feedback_list_key="current_feedbacks"
)

# AB mode
feedback_stats_a = SessionFeedbackManager.get_feedback_stats(
    experiment_a_id,
    total_messages=len(messages_a_df),
    feedback_list_key="feedbacks_a"
)
feedback_stats_b = SessionFeedbackManager.get_feedback_stats(
    experiment_b_id,
    total_messages=len(messages_b_df),
    feedback_list_key="feedbacks_b"
)
```

**Metrics Displayed**:
- Overall: Total messages, rejection rate, feedback count
- Stage-by-Stage: Performance breakdown per stage
- Rejection Reasons: Pie charts and bar charts
- AB Comparison: Side-by-side metrics with winner determination

**Export Options**:
- Export current messages to CSV
- Export current feedback to CSV
- Export analytics summary to CSV

**Important**: Analytics page shows ONLY the current in-memory experiment. For historical data, use Historical Analytics.

---

### Page 4: Historical Analytics

**Purpose**: Track all past experiments and analyze trends from Snowflake

**Architecture**: Button-triggered Snowflake queries

#### Key Features

**Load Button**:
```python
if st.button("📊 Load Historical Data from Snowflake"):
    session = create_snowflake_session()
    db_manager = UIDatabaseManager(session)

    # Load experiment summary with JOIN
    experiments_df = db_manager.get_experiment_summary(brand=brand)

    st.session_state['historical_experiments'] = experiments_df
    st.session_state['historical_data_loaded'] = True

    session.close()
```

**SQL Query Example**:
```sql
SELECT
    m.EXPERIMENT_ID,
    m.CONFIG_NAME,
    m.CAMPAIGN_NAME,
    m.BRAND,
    MIN(m.START_TIME) as start_time,
    SUM(m.TOTAL_MESSAGES) as total_messages,
    MAX(m.TOTAL_USERS) as total_users,
    COUNT(DISTINCT m.STAGE) as total_stages,
    COUNT(f.FEEDBACK_TYPE) as total_rejects,
    (COUNT(f.FEEDBACK_TYPE) * 100.0 / NULLIF(SUM(m.TOTAL_MESSAGES), 0)) as rejection_rate
FROM MESSAGING_SYSTEM_V2.UI.EXPERIMENT_METADATA m
LEFT JOIN MESSAGING_SYSTEM_V2.UI.FEEDBACKS f
    ON m.EXPERIMENT_ID = f.EXPERIMENT_ID
WHERE m.BRAND = :brand
GROUP BY m.EXPERIMENT_ID, m.CONFIG_NAME, m.CAMPAIGN_NAME, m.BRAND
ORDER BY start_time DESC
```

**Visualizations**:
- Experiments summary table
- Rejection rate trend over time (line chart)
- Performance comparison by configuration (bar chart)
- Best/worst performing configs

**Filters**:
- Date range filtering
- Automatic refresh button

**Export**:
- Export summary to CSV
- Note: Detailed feedback export coming soon (use SQL queries for now)

---

## 🔧 Utility Modules

### utils/db_manager.py - UIDatabaseManager

**Purpose**: All Snowflake database operations

**Key Methods**:
```python
class UIDatabaseManager:
    def __init__(self, session: Session):
        """Initialize with Snowflake session."""

    def save_config(self, config_name, config_data, brand):
        """Save config with auto-versioning."""

    def load_config(self, config_name, brand, version=None):
        """Load specific config version."""

    def store_experiment_metadata(self, metadata: dict):
        """Insert metadata record."""

    def store_feedback(self, feedback: dict):
        """Insert feedback record."""

    def get_experiment_summary(self, brand=None, start_date=None, end_date=None):
        """Get aggregated experiment metrics with JOIN."""

    def close(self):
        """Close Snowflake session."""
```

**Usage Pattern**:
```python
# Always use context-like pattern
session = create_snowflake_session()
db_manager = UIDatabaseManager(session)

try:
    # Do operations
    db_manager.save_config(...)
    db_manager.store_feedback(...)
finally:
    db_manager.close()  # or session.close()
```

---

### utils/config_manager.py - ConfigManager

**Purpose**: Configuration loading and caching

**Key Methods**:
```python
class ConfigManager:
    def __init__(self, session: Session):
        """Initialize with Snowflake session."""

    def load_configs_from_snowflake(self, brand: str) -> Dict:
        """Load all configs for brand, returns {name: config_data}."""

    def get_latest_version(self, config_name: str, brand: str) -> int:
        """Get latest version number."""
```

**Caching Strategy**:
- Configs loaded once on app startup
- Cached in `st.session_state.configs_cache`
- Format: `{"config_name": {...config_data...}, ...}`
- No re-querying Snowflake during session

---

### utils/experiment_runner.py - ExperimentRunner

**Purpose**: Execute experiments with proper session management

**Key Methods**:
```python
class ExperimentRunner:
    def run_single_experiment(
        self, config, sampled_users_df, experiment_id,
        create_session_func, progress_container
    ):
        """Run one experiment, all stages sequentially."""

    def run_ab_test_parallel(
        self, config_a, config_b, sampled_users_df,
        experiment_a_id, experiment_b_id, create_session_func
    ):
        """Run two experiments in parallel threads."""
```

**Thread-Safe Design**:
- Each thread gets own Snowflake session
- Progress updates handled safely:
  - Main thread: Full Streamlit UI
  - Worker threads: Console logging with dummy UI objects
- No Streamlit context errors

**Implementation**:
```python
# Thread function
def run_experiment(exp_key, config, exp_id):
    try:
        success, data, metadata = self.run_single_experiment(
            config=config,
            sampled_users_df=users_df,
            experiment_id=exp_id,
            create_session_func=create_session_func,
            progress_container=None  # None = threaded mode
        )
        results[exp_key] = {'success': success, 'ui_log_data': data, 'metadata': metadata}
    except Exception as e:
        results[exp_key] = {'success': False, 'error': str(e)}

# Start threads
thread_a = threading.Thread(target=run_experiment, args=('a', config_a, exp_a_id))
thread_b = threading.Thread(target=run_experiment, args=('b', config_b, exp_b_id))
thread_a.start()
thread_b.start()
thread_a.join()
thread_b.join()
```

---

### utils/session_feedback_manager.py - SessionFeedbackManager

**Purpose**: In-memory feedback management

**Static Methods** (no instance needed):
```python
@staticmethod
def add_feedback(experiment_id, user_id, stage, feedback_type,
                 rejection_reason, rejection_text, message_header,
                 message_body, campaign_name, brand, config_name,
                 feedback_list_key):
    """Add feedback to session_state list."""

@staticmethod
def get_feedback_stats(experiment_id, total_messages, feedback_list_key):
    """Calculate aggregate stats from feedback list."""

@staticmethod
def get_stage_feedback_stats(experiment_id, messages_df, feedback_list_key):
    """Calculate per-stage stats."""

@staticmethod
def get_rejection_reason_label(reason_key):
    """Map reason key to display label."""
```

**Feedback List Keys**:
- `"current_feedbacks"` - Single mode
- `"feedbacks_a"` - AB mode experiment A
- `"feedbacks_b"` - AB mode experiment B

**Usage**:
```python
# Add feedback
SessionFeedbackManager.add_feedback(
    experiment_id="drumeo_20260114_1234",
    user_id=12345,
    stage=1,
    feedback_type="reject",
    rejection_reason="poor_header",
    rejection_text="Too generic",
    message_header="Your next lesson 👇",
    message_body="Check it out...",
    campaign_name="Re-engagement",
    brand="drumeo",
    config_name="drumeo_re_engagement_test",
    feedback_list_key="current_feedbacks"
)

# Get stats
stats = SessionFeedbackManager.get_feedback_stats(
    experiment_id="drumeo_20260114_1234",
    total_messages=100,
    feedback_list_key="current_feedbacks"
)
# Returns: {'total_feedback': 10, 'total_rejects': 10,
#           'reject_rate': 10.0, 'rejection_reasons': {...}}
```

---

## 🎨 Brand Theming

Theming automatically adjusts based on selected brand:

| Brand    | Primary Color | Sidebar BG | Accent   | Emoji |
|----------|---------------|------------|----------|-------|
| Base     | Gold          | Dark Gold  | Gold     | 🎵    |
| Drumeo   | Light Blue    | Dark Blue  | Blue     | 🥁    |
| Pianote  | Light Red     | Dark Red   | Red      | 🎹    |
| Guitareo | Light Green   | Dark Green | Green    | 🎸    |
| Singeo   | Light Purple  | Dark Purple| Purple   | 🎤    |

**Implementation**: `utils/theme.py`

```python
def get_brand_theme(brand):
    """Returns theme dictionary."""

def apply_theme(brand):
    """Applies CSS via st.markdown."""

def get_brand_emoji(brand):
    """Returns brand emoji."""
```

---

## 🔍 Key Implementation Details

### Configuration File Structure

```json
{
  "brand": "drumeo",
  "campaign_type": "re_engagement",
  "campaign_name": "UI-Test-Campaign",
  "campaign_instructions": "Keep messages encouraging and motivational.",
  "1": {
    "stage": 1,
    "model": "gemini-2.5-flash-lite",
    "personalization": true,
    "involve_recsys_result": true,
    "recsys_contents": ["workout", "course", "quick_tips"],
    "specific_content_id": null,
    "segment_info": "Students inactive for 3+ days",
    "instructions": "",
    "sample_examples": "Header: Your next lesson 👇\nMessage: Check it out!",
    "identifier_column": "user_id",
    "platform": "push"
  },
  "2": {
    "stage": 2,
    "model": "gpt-4o-mini",
    "personalization": true,
    "involve_recsys_result": true,
    "recsys_contents": ["song"],
    "specific_content_id": 12345,
    "segment_info": "Students inactive for 7+ days",
    "instructions": "Focus on easy songs",
    "sample_examples": "Header: Let's jam! 🎸\nMessage: Try this song!",
    "identifier_column": "user_id",
    "platform": "push"
  }
}
```

### Rejection Reason Categories

```python
REJECTION_REASONS = {
    "poor_header": "Poor Header",
    "poor_body": "Poor Body/Content",
    "grammar_issues": "Grammar Issues",
    "emoji_problems": "Emoji Problems",
    "recommendation_issues": "Recommendation Issues",
    "wrong_information": "Wrong/Inaccurate Information",
    "tone_issues": "Tone Issues",
    "similarity": "Similar To Previous Header/Messages",
    "other": "Other"
}
```

### Environment Variables Required

```bash
# Snowflake
SNOWFLAKE_USER=your_user
SNOWFLAKE_PASSWORD=your_password
SNOWFLAKE_ACCOUNT=your_account
SNOWFLAKE_ROLE=your_role
SNOWFLAKE_DATABASE=MESSAGING_SYSTEM_V2
SNOWFLAKE_WAREHOUSE=your_warehouse
SNOWFLAKE_SCHEMA=UI

# LLM APIs
OPENAI_API_KEY=sk-...
GOOGLE_API_KEY=AIza...
```

---

## 🛠️ Development Guide

### Adding a New Page

1. Create `pages/{number}_{Name}.py`
2. Follow standard structure:
```python
import streamlit as st
import os
from pathlib import Path
from dotenv import load_dotenv

# Load .env
env_path = Path(__file__).parent.parent.parent / '.env'
if env_path.exists():
    load_dotenv(env_path)

from utils.auth import check_authentication
from utils.theme import apply_theme, get_brand_emoji

st.set_page_config(page_title="New Page", page_icon="🆕", layout="wide")

if not check_authentication():
    st.error("🔒 Please login first")
    st.stop()

if "selected_brand" not in st.session_state:
    st.error("⚠️ Please select a brand first")
    st.stop()

brand = st.session_state.selected_brand
apply_theme(brand)

# Your content here
```
3. Update `app.py` navigation
4. Test with all brands

### Adding a New Feedback Category

1. Update `utils/session_feedback_manager.py`:
```python
REJECTION_REASONS = {
    # ... existing
    "new_reason": "New Reason Label"
}
```
2. Automatically appears in UI

### Adding a New Brand

1. Create `data/UI_users/{brand}_users.csv` (100 users, must have `USER_ID` column)
2. Add to `utils/theme.py`:
```python
BRAND_THEMES["newbrand"] = {
    "primary": "#COLOR",
    "accent": "#COLOR",
    "sidebar_bg": "#DARK_COLOR",
    "text": "#FFFFFF"
}
BRAND_EMOJIS["newbrand"] = "🆕"
```
3. Update `app.py`:
```python
brands = ["drumeo", "pianote", "guitareo", "singeo", "newbrand"]
brand_labels["newbrand"] = "🆕 New Brand"
```
4. Create default config in Snowflake or via UI

### Debugging Tips

**Common Issues**:

1. **"No messages found"** in Message Viewer:
   - Check `st.session_state.ui_log_data` exists
   - Verify generation completed in Campaign Builder
   - Look for errors in terminal

2. **Snowflake connection errors**:
   - Verify `.env` file exists and is loaded
   - Check credentials are correct
   - Test connection: `create_snowflake_session()`

3. **AB test AttributeError**:
   - Fixed in latest version
   - Ensure ExperimentRunner uses thread-safe progress handling

4. **Config save errors with quotes**:
   - Fixed: Now uses `write_pandas()` instead of raw SQL
   - Handles JSON with apostrophes correctly

5. **Feedback not in Analytics**:
   - Check `st.session_state.current_feedbacks` has data
   - Verify correct feedback_list_key used
   - Check experiment_id matches

**Debugging Code**:
```python
# Debug session state
with st.expander("Debug Info"):
    st.write("Session State Keys:", list(st.session_state.keys()))
    if 'ui_log_data' in st.session_state:
        st.write("Messages shape:", st.session_state.ui_log_data.shape)
    if 'current_feedbacks' in st.session_state:
        st.write("Feedback count:", len(st.session_state.current_feedbacks))
```

---

## 🎯 Key Design Decisions

### 1. In-Memory + Cloud Hybrid
**Decision**: Use session_state for active data, Snowflake for persistence
**Rationale**:
- Fast in-memory operations
- No local file dependencies (HuggingFace ready)
- Scalable historical storage
- Clean separation: current vs. historical

### 2. One-Click Storage
**Decision**: Single "Store Results" button to persist everything
**Rationale**:
- Simple user workflow
- Explicit persistence action
- User controls when data is saved
- No auto-save surprises

### 3. Config Caching
**Decision**: Load all configs once, cache in session_state
**Rationale**:
- Reduces Snowflake queries
- Faster config switching
- Session-scoped cache (fresh on page load)
- No stale data issues

### 4. Thread-Safe AB Testing
**Decision**: Separate Snowflake sessions per thread, console logging
**Rationale**:
- Prevents session conflicts
- Streamlit UI only in main thread
- Clean error handling
- Production-ready parallel execution

### 5. Versioned Configurations
**Decision**: Auto-increment version on every config save
**Rationale**:
- Full audit trail
- Can rollback to previous versions
- Supports experimentation
- No data loss

### 6. Button-Triggered Historical Loading
**Decision**: Historical Analytics loads on button click, not auto
**Rationale**:
- User controls when to query Snowflake
- Avoids unnecessary queries
- Faster page load
- Clear user action

### 7. SessionFeedbackManager Static Methods
**Decision**: All methods static, no instance needed
**Rationale**:
- Simpler API
- Works directly with session_state
- No state to manage
- Cleaner code

---

## 🚀 Deployment Guide

### HuggingFace Spaces Deployment

**Requirements**:
- No local file dependencies ✅
- Environment variables for secrets ✅
- Snowflake connectivity ✅
- CSV files in repo (data/UI_users/) ✅

**Steps**:
1. Push code to GitHub/HuggingFace repo
2. Include `data/UI_users/` CSV files
3. Set environment variables in Space settings:
   - All SNOWFLAKE_* variables
   - All API keys
4. Run: `streamlit run app.py`
5. Verify Snowflake connection works

**Files to Exclude**:
- `.env` (use Space secrets instead)
- Local cache directories
- Test data

---

## 📞 Support & Resources

**Contact**:
- Technical Support: danial@musora.com

**Related Documentation**:
- Main System: `ai_messaging_system_v2/README.md`
- UI Mode Guide: `ai_messaging_system_v2/UI_MODE_GUIDE.md`
- Implementation Details: `visualization/IMPLEMENTATION_COMPLETE.md`
- Refactoring Guide: `visualization/ARCHITECTURE_REFACTOR_GUIDE.md`

**Useful Links**:
- Streamlit Documentation: https://docs.streamlit.io
- Snowflake Python Connector: https://docs.snowflake.com/en/developer-guide/python-connector/python-connector
- Plotly Charts: https://plotly.com/python/

---

## 📊 System Status

**Completion**: 100% ✅

**Completed Components**:
1. ✅ Database Layer (db_manager.py)
2. ✅ Config Manager (config_manager.py)
3. ✅ Session Feedback Manager (session_feedback_manager.py)
4. ✅ Experiment Runner (experiment_runner.py)
5. ✅ App.py - Authentication & config loading
6. ✅ Campaign Builder - Generation & AB testing
7. ✅ Message Viewer - Viewing & feedback
8. ✅ Analytics - Current experiment metrics
9. ✅ Historical Analytics - Snowflake integration

**Recent Fixes**:
- ✅ Configuration save error (JSON escaping) - Fixed with `write_pandas()`
- ✅ AB testing `AttributeError: enter` - Fixed with thread-safe design
- ✅ Historical Analytics Snowflake connection - Fixed to use `.env`

**Ready For**:
- ✅ Production use
- ✅ HuggingFace deployment
- ✅ End-to-end testing
- ✅ Team onboarding

---

**Built with ❤️ for the Musora team**

**Last Updated**: 2026-01-14
**Version**: 2.0 (Refactored Architecture)