docs/archive/PHASE_3_IMPLEMENTATION_GUIDE.md

# SPARKNET Phase 3: Production Web UI Implementation Guide

## 🎉 Phase 3 Progress: Backend Complete!

**Status**: FastAPI Backend ✅ COMPLETE | Frontend 🚧 IN PROGRESS

---

## ✅ Completed: FastAPI Backend

### Files Created

1. **`api/main.py`** (~150 lines)
   - FastAPI application with lifecycle management
   - CORS middleware for frontend integration
   - Auto-initialization of SPARKNET components
   - Health check endpoints
   - OpenAPI documentation at `/api/docs`

2. **`api/routes/patents.py`** (~200 lines)
   - POST `/api/patents/upload` - Upload patent PDF
   - GET `/api/patents/{id}` - Get patent metadata
   - GET `/api/patents/` - List all patents with pagination
   - DELETE `/api/patents/{id}` - Delete patent
   - GET `/api/patents/{id}/download` - Download original PDF

3. **`api/routes/workflows.py`** (~300 lines)
   - POST `/api/workflows/execute` - Start Patent Wake-Up workflow
   - GET `/api/workflows/{id}` - Get workflow status
   - WS `/api/workflows/{id}/stream` - WebSocket for real-time updates
   - GET `/api/workflows/` - List all workflows
   - GET `/api/workflows/{id}/brief/download` - Download valorization brief

4. **`api/requirements.txt`**
   - FastAPI, Uvicorn, WebSockets, Pydantic dependencies

---

## 🚀 Quick Start: Test the API

### Step 1: Install Dependencies

```bash
cd /home/mhamdan/SPARKNET

# Activate conda environment
conda activate agentic-ai

# Install FastAPI dependencies
pip install fastapi uvicorn python-multipart websockets
```

### Step 2: Start the API Server

```bash
# Make sure Ollama is running
# (Should already be running from background processes)

# Start FastAPI
python -m api.main
```

The API will be available at:
- **API**: http://localhost:8000
- **Docs**: http://localhost:8000/api/docs (Interactive OpenAPI documentation)
- **Health**: http://localhost:8000/api/health

### Step 3: Test with curl

```bash
# Health check
curl http://localhost:8000/api/health

# Upload a patent
curl -X POST http://localhost:8000/api/patents/upload \
  -F "file=@Dataset/your_patent.pdf"

# Start workflow (replace PATENT_ID)
curl -X POST http://localhost:8000/api/workflows/execute \
  -H "Content-Type: application/json" \
  -d '{"patent_id": "PATENT_ID"}'

# Check workflow status (replace WORKFLOW_ID)
curl http://localhost:8000/api/workflows/WORKFLOW_ID
```

---

## 📊 API Endpoints Reference

### Patents Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/patents/upload` | Upload patent PDF (max 50MB) |
| GET | `/api/patents/{id}` | Get patent metadata |
| GET | `/api/patents/` | List all patents (supports pagination) |
| DELETE | `/api/patents/{id}` | Delete patent |
| GET | `/api/patents/{id}/download` | Download original PDF |

**Example Upload Response**:
```json
{
  "patent_id": "550e8400-e29b-41d4-a716-446655440000",
  "filename": "ai_drug_discovery.pdf",
  "size": 2457600,
  "uploaded_at": "2025-11-04T20:00:00.000Z",
  "message": "Patent uploaded successfully"
}
```

### Workflows Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/workflows/execute` | Start Patent Wake-Up workflow |
| GET | `/api/workflows/{id}` | Get workflow status and results |
| WS | `/api/workflows/{id}/stream` | Real-time WebSocket updates |
| GET | `/api/workflows/` | List all workflows (supports pagination) |
| GET | `/api/workflows/{id}/brief/download` | Download valorization brief PDF |

**Example Workflow Response**:
```json
{
  "id": "workflow-uuid",
  "patent_id": "patent-uuid",
  "status": "running",
  "progress": 45,
  "current_step": "market_analysis",
  "started_at": "2025-11-04T20:01:00.000Z",
  "completed_at": null,
  "result": null
}
```

**Workflow States**:
- `queued` - Waiting to start
- `running` - Currently executing
- `completed` - Successfully finished
- `failed` - Error occurred

---

## 🔄 WebSocket Real-Time Updates

The WebSocket endpoint provides live progress updates:

```javascript
// JavaScript example
const ws = new WebSocket('ws://localhost:8000/api/workflows/{workflow_id}/stream');

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  console.log(`Status: ${data.status}, Progress: ${data.progress}%`);

  if (data.status === 'completed') {
    // Workflow finished, display results
    console.log('Results:', data.result);
  }
};
```

---

## 🎨 Next Steps: Frontend Implementation

### Option 1: Build Next.js Frontend (Recommended)

**Technologies**:
- Next.js 14 with App Router
- TypeScript for type safety
- Tailwind CSS for styling
- shadcn/ui for components
- Framer Motion for animations

**Setup Commands**:
```bash
# Create Next.js app
cd /home/mhamdan/SPARKNET
npx create-next-app@latest frontend --typescript --tailwind --app

cd frontend

# Install dependencies
npm install @radix-ui/react-dialog @radix-ui/react-progress
npm install framer-motion recharts lucide-react
npm install class-variance-authority clsx tailwind-merge

# Install shadcn/ui
npx shadcn-ui@latest init
npx shadcn-ui@latest add button card input progress badge tabs dialog
```

**Key Pages to Build**:
1. **Home Page** (`app/page.tsx`) - Landing page with features
2. **Upload Page** (`app/upload/page.tsx`) - Drag-and-drop patent upload
3. **Workflow Page** (`app/workflow/[id]/page.tsx`) - Live progress tracking
4. **Results Page** (`app/results/[id]/page.tsx`) - Beautiful result displays

### Option 2: Simple HTML + JavaScript Frontend

For quick testing, create a simple HTML interface:

```html
<!-- frontend/index.html -->
<!DOCTYPE html>
<html>
<head>
    <title>SPARKNET</title>
    <script src="https://cdn.tailwindcss.com"></script>
</head>
<body class="bg-gray-50">
    <div class="container mx-auto p-8">
        <h1 class="text-4xl font-bold mb-8">SPARKNET - Patent Analysis</h1>

        <!-- Upload Form -->
        <div class="bg-white p-6 rounded-lg shadow mb-8">
            <h2 class="text-2xl font-semibold mb-4">Upload Patent</h2>
            <input type="file" id="fileInput" accept=".pdf" class="mb-4">
            <button onclick="uploadPatent()" class="bg-blue-600 text-white px-6 py-2 rounded">
                Upload & Analyze
            </button>
        </div>

        <!-- Results -->
        <div id="results" class="bg-white p-6 rounded-lg shadow hidden">
            <h2 class="text-2xl font-semibold mb-4">Analysis Results</h2>
            <div id="resultsContent"></div>
        </div>
    </div>

    <script>
        async function uploadPatent() {
            const fileInput = document.getElementById('fileInput');
            const file = fileInput.files[0];

            if (!file) {
                alert('Please select a file');
                return;
            }

            // Upload patent
            const formData = new FormData();
            formData.append('file', file);

            const uploadRes = await fetch('http://localhost:8000/api/patents/upload', {
                method: 'POST',
                body: formData
            });

            const upload = await uploadRes.json();
            console.log('Uploaded:', upload);

            // Start workflow
            const workflowRes = await fetch('http://localhost:8000/api/workflows/execute', {
                method: 'POST',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({ patent_id: upload.patent_id })
            });

            const workflow = await workflowRes.json();
            console.log('Workflow started:', workflow);

            // Monitor progress
            monitorWorkflow(workflow.workflow_id);
        }

        async function monitorWorkflow(workflowId) {
            const ws = new WebSocket(`ws://localhost:8000/api/workflows/${workflowId}/stream`);

            ws.onmessage = (event) => {
                const data = JSON.parse(event.data);
                console.log('Progress:', data.progress + '%');

                if (data.status === 'completed') {
                    displayResults(data.result);
                }
            };
        }

        function displayResults(result) {
            const resultsDiv = document.getElementById('results');
            const contentDiv = document.getElementById('resultsContent');

            resultsDiv.classList.remove('hidden');

            contentDiv.innerHTML = `
                <p><strong>Quality Score:</strong> ${(result.quality_score * 100).toFixed(0)}%</p>
                <p><strong>TRL Level:</strong> ${result.document_analysis?.trl_level}/9</p>
                <p><strong>Market Opportunities:</strong> ${result.market_analysis?.opportunities?.length || 0}</p>
                <p><strong>Partner Matches:</strong> ${result.matches?.length || 0}</p>
            `;
        }
    </script>
</body>
</html>
```

---

## 🧪 Testing the Backend

### Manual Testing with OpenAPI Docs

1. Start the API: `python -m api.main`
2. Open browser: http://localhost:8000/api/docs
3. Try the interactive endpoints:
   - Upload a patent
   - Start a workflow
   - Check workflow status

### Automated Testing Script

```bash
# test_api.sh
#!/bin/bash

echo "Testing SPARKNET API..."

# Health check
echo "\n1. Health Check"
curl -s http://localhost:8000/api/health | json_pp

# Upload patent (replace with actual file path)
echo "\n2. Uploading Patent"
UPLOAD_RESULT=$(curl -s -X POST http://localhost:8000/api/patents/upload \
  -F "file=@Dataset/sample_patent.pdf")
echo $UPLOAD_RESULT | json_pp

# Extract patent ID
PATENT_ID=$(echo $UPLOAD_RESULT | jq -r '.patent_id')
echo "Patent ID: $PATENT_ID"

# Start workflow
echo "\n3. Starting Workflow"
WORKFLOW_RESULT=$(curl -s -X POST http://localhost:8000/api/workflows/execute \
  -H "Content-Type: application/json" \
  -d "{\"patent_id\": \"$PATENT_ID\"}")
echo $WORKFLOW_RESULT | json_pp

# Extract workflow ID
WORKFLOW_ID=$(echo $WORKFLOW_RESULT | jq -r '.workflow_id')
echo "Workflow ID: $WORKFLOW_ID"

# Monitor workflow
echo "\n4. Monitoring Workflow (checking every 5 seconds)"
while true; do
  STATUS=$(curl -s http://localhost:8000/api/workflows/$WORKFLOW_ID | jq -r '.status')
  PROGRESS=$(curl -s http://localhost:8000/api/workflows/$WORKFLOW_ID | jq -r '.progress')

  echo "Status: $STATUS, Progress: $PROGRESS%"

  if [ "$STATUS" = "completed" ] || [ "$STATUS" = "failed" ]; then
    break
  fi

  sleep 5
done

echo "\n5. Final Results"
curl -s http://localhost:8000/api/workflows/$WORKFLOW_ID | jq '.result'
```

---

## 📦 Deployment with Docker

### Dockerfile for API

```dockerfile
# Dockerfile.api
FROM python:3.10-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    g++ \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements
COPY requirements.txt api/requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt -r api/requirements.txt

# Copy application
COPY . .

# Expose port
EXPOSE 8000

# Run API
CMD ["python", "-m", "api.main"]
```

### Docker Compose

```yaml
# docker-compose.yml
version: '3.8'

services:
  api:
    build:
      context: .
      dockerfile: Dockerfile.api
    ports:
      - "8000:8000"
    volumes:
      - ./uploads:/app/uploads
      - ./outputs:/app/outputs
      - ./data:/app/data
    environment:
      - OLLAMA_HOST=http://host.docker.internal:11434
    restart: unless-stopped
```

**Start with Docker**:
```bash
docker-compose up --build
```

---

## 🎯 Current Status Summary

### ✅ Completed

1. **FastAPI Backend** - Full RESTful API with WebSocket support
2. **Patent Upload** - File validation, storage, metadata tracking
3. **Workflow Execution** - Background task processing
4. **Real-Time Updates** - WebSocket streaming
5. **Result Retrieval** - Complete workflow results API
6. **API Documentation** - Auto-generated OpenAPI docs

### 🚧 In Progress

1. **Frontend Development** - Next.js app (ready to start)
2. **UI Components** - Beautiful React components (pending)
3. **Dataset Testing** - Batch processing script (pending)

### 📋 Next Steps

1. **Test the Backend API** - Ensure all endpoints work correctly
2. **Set up Next.js Frontend** - Modern React application
3. **Build UI Components** - Beautiful, animated components
4. **Integrate Frontend with API** - Connect all the pieces
5. **Test with Dataset** - Process all patents in Dataset/
6. **Deploy** - Docker containers for production

---

## 💡 Development Tips

### Running API in Development

```bash
# With auto-reload
uvicorn api.main:app --reload --host 0.0.0.0 --port 8000

# With custom log level
uvicorn api.main:app --log-level debug
```

### Debugging

- Check logs in terminal where API is running
- Use OpenAPI docs for interactive testing: http://localhost:8000/api/docs
- Monitor workflow state in real-time with WebSocket
- Check file uploads in `uploads/patents/` directory
- Check generated briefs in `outputs/` directory

### Environment Variables

Create `.env` file for configuration:
```env
OLLAMA_HOST=http://localhost:11434
API_HOST=0.0.0.0
API_PORT=8000
MAX_UPLOAD_SIZE=52428800  # 50MB
```

---

## 🎬 Ready for Phase 3B: Frontend!

The backend is complete and ready to serve the frontend. Next, we'll build a beautiful web interface that leverages all these API endpoints.

**What we'll build next**:
1. **Modern UI** with Next.js + Tailwind
2. **Drag-and-drop Upload** - Beautiful file upload experience
3. **Live Progress Tracking** - Real-time workflow visualization
4. **Interactive Results** - Charts, cards, and detailed displays
5. **Responsive Design** - Works on all devices

The foundation is solid - now let's make it beautiful! 🚀