| # Testing Guide: DocGenie API |
|
|
| Complete guide for testing the document generation API endpoints with Google Drive integration. |
|
|
| ## Table of Contents |
|
|
| 1. [Prerequisites](#prerequisites) |
| 2. [Quick Start](#quick-start) |
| 3. [Getting Google Drive Token](#getting-google-drive-token) |
| 4. [Testing Async API](#testing-async-api) |
| 5. [Testing Sync PDF API](#testing-sync-pdf-api) |
| 6. [Manual Testing with cURL](#manual-testing-with-curl) |
| 7. [Frontend Integration Example](#frontend-integration-example) |
| 8. [Troubleshooting](#troubleshooting) |
|
|
| --- |
|
|
| ## Prerequisites |
|
|
| ### 1. Start Required Services |
|
|
| ```bash |
| # Terminal 1: Start Redis |
| |
| ## Option A: Local Redis (Recommended for Development) |
| # Install Redis (Ubuntu/Debian) |
| sudo apt-get update && sudo apt-get install redis-server -y |
| sudo systemctl start redis-server |
| sudo systemctl enable redis-server |
| |
| # Verify Redis is running |
| redis-cli ping # Should return "PONG" |
| |
| ## Option B: Docker (if Docker is installed) |
| # docker run -d -p 6379:6379 --name redis redis:7-alpine |
| |
| # Terminal 2: Start FastAPI Server |
| cd docgenie/api |
| python main.py |
| |
| # Terminal 3: Start RQ Worker |
| cd docgenie/api |
| ./start_worker.sh |
| ``` |
|
|
| ### 2. Configure Environment |
|
|
| Make sure your `api/.env` file has: |
|
|
| ```bash |
| # Required |
| ANTHROPIC_API_KEY=your_claude_api_key |
| SUPABASE_URL=https://your-project.supabase.co |
| SUPABASE_KEY=your_supabase_key |
| REDIS_URL=redis://localhost:6379/0 |
| |
| # Optional (for token refresh) |
| GOOGLE_CLIENT_ID=your_client_id.apps.googleusercontent.com |
| GOOGLE_CLIENT_SECRET=your_client_secret |
| ``` |
|
|
| ### 3. Create Supabase Tables |
|
|
| Run the SQL from [DEPLOYMENT.md](DEPLOYMENT.md#32-create-database-schema) in your Supabase SQL Editor. |
|
|
| --- |
|
|
| ## Quick Start |
|
|
| ### Option 1: Using Test Script (Easiest) |
|
|
| ```bash |
| # Get Google Drive token first (one-time setup) |
| python api/test_get_google_token.py \ |
| --client-id YOUR_CLIENT_ID \ |
| --client-secret YOUR_CLIENT_SECRET |
| |
| # Copy the access token, then run test |
| python api/test_async_api.py --google-token YOUR_ACCESS_TOKEN |
| ``` |
|
|
| ### Option 2: Using OAuth Playground (Quick Test) |
|
|
| 1. Go to [OAuth Playground](https://developers.google.com/oauthplayground/) |
| 2. Configure with your credentials |
| 3. Get access token |
| 4. Run test script with token |
|
|
| --- |
|
|
| ## Getting Google Drive Token |
|
|
| ### Method 1: Using Helper Script (Recommended) |
|
|
| Our helper script automates the OAuth flow: |
|
|
| ```bash |
| cd docgenie/api |
| |
| python test_get_google_token.py \ |
| --client-id YOUR_GOOGLE_CLIENT_ID \ |
| --client-secret YOUR_GOOGLE_CLIENT_SECRET |
| ``` |
|
|
| **What it does:** |
| 1. Opens browser for Google authorization |
| 2. Starts local server on port 8080 for callback |
| 3. Exchanges authorization code for tokens |
| 4. Displays access token and refresh token |
|
|
| **Output:** |
| ``` |
| Access Token: ya29.a0AfH6SMBx... |
| Refresh Token: 1//0gw... |
| ``` |
|
|
| ### Method 2: OAuth Playground (No Code) |
|
|
| 1. **Go to**: https://developers.google.com/oauthplayground/ |
|
|
| 2. **Configure Credentials**: |
| - Click gear icon (β) in top right |
| - Check "Use your own OAuth credentials" |
| - Enter your Client ID and Client Secret |
|
|
| 3. **Authorize API**: |
| - In left panel, scroll to "Drive API v3" |
| - Select: `https://www.googleapis.com/auth/drive.file` |
| - Click "Authorize APIs" |
| - Sign in with your Google account |
|
|
| 4. **Get Token**: |
| - Click "Exchange authorization code for tokens" |
| - Copy the "Access token" value |
|
|
| ### Method 3: Manual cURL (For Advanced Users) |
|
|
| **Step 1: Get Authorization Code** |
|
|
| Open this URL in browser (replace YOUR_CLIENT_ID): |
|
|
| ``` |
| https://accounts.google.com/o/oauth2/v2/auth?client_id=YOUR_CLIENT_ID&redirect_uri=http://localhost:8080&response_type=code&scope=https://www.googleapis.com/auth/drive.file&access_type=offline&prompt=consent |
| ``` |
|
|
| **Step 2: Exchange Code for Token** |
|
|
| After authorization, you'll be redirected to: |
| ``` |
| http://localhost:8080/?code=AUTHORIZATION_CODE |
| ``` |
|
|
| Exchange the code: |
|
|
| ```bash |
| curl -X POST https://oauth2.googleapis.com/token \ |
| -d "code=AUTHORIZATION_CODE" \ |
| -d "client_id=YOUR_CLIENT_ID" \ |
| -d "client_secret=YOUR_CLIENT_SECRET" \ |
| -d "redirect_uri=http://localhost:8080" \ |
| -d "grant_type=authorization_code" |
| ``` |
|
|
| Response: |
| ```json |
| { |
| "access_token": "ya29.a0AfH6SMBx...", |
| "refresh_token": "1//0gw...", |
| "expires_in": 3600, |
| "token_type": "Bearer" |
| } |
| ``` |
|
|
| --- |
|
|
| ## Testing Async API |
|
|
| The async API (`/generate/async`) is optimized for batch processing with 50% cost savings. Jobs are queued and processed in the background, with status polling. |
|
|
| ### Full Automated Test |
|
|
| ```bash |
| cd docgenie/api |
| |
| # Set token as environment variable |
| export GOOGLE_DRIVE_TOKEN="ya29.a0AfH6SMBx..." |
| |
| # Run test (generates 2 documents by default) |
| python test_async_api.py |
| |
| # Or pass token directly |
| python test_async_api.py --google-token "ya29.a0AfH6SMBx..." |
| ``` |
|
|
| **Test Flow:** |
| 1. β Health check |
| 2. β Submit async job |
| 3. β Poll status (every 30 seconds) |
| 4. β List user jobs |
| 5. β Display Google Drive link |
|
|
| **Expected Output:** |
| ``` |
| ================================================================================ |
| ASYNC API TEST SUITE |
| ================================================================================ |
| Base URL: http://localhost:8000 |
| User ID: 1 |
| Documents to Generate: 2 |
| ================================================================================ |
|
|
| ============================================================ |
| 1. Testing API Health |
| ============================================================ |
| β API is healthy: {'status': 'healthy', 'version': '1.0.0'} |
|
|
| ============================================================ |
| 2. Submitting Async Job |
| ============================================================ |
| Payload: |
| User ID: 1 |
| Seed Images: 1 |
| Num Solutions: 2 |
| Google Token: ya29.a0AfH6SMBx... |
|
|
| β Job submitted successfully! |
| Request ID: 550e8400-e29b-41d4-a716-446655440000 |
| Status: queued |
| Estimated Time: 10 minutes |
| Poll URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status |
|
|
| ============================================================ |
| 3. Polling Job Status |
| ============================================================ |
| Polling every 30 seconds (max 60 attempts) |
| Status flow: queued β processing β generating β completed/failed |
|
|
| [12:00:00] Poll 1/60: QUEUED |
| [12:00:30] Poll 2/60: PROCESSING - Creating batch request... |
| [12:01:00] Poll 3/60: GENERATING - Batch submitted to Claude... |
| [12:08:30] Poll 17/60: GENERATING - Polling batch status... |
| [12:15:00] Poll 30/60: COMPLETED |
|
|
| ============================================================ |
| β JOB COMPLETED! |
| ============================================================ |
| Download URL: https://drive.google.com/file/d/abc123xyz/view?usp=sharing |
| File Size: 15.4 MB |
| Document Count: 2 |
| Created: 2026-02-28T12:00:00Z |
| Completed: 2026-02-28T12:15:00Z |
|
|
| ================================================================================ |
| TEST SUMMARY |
| ================================================================================ |
| β ALL TESTS PASSED! |
|
|
| Your documents are available at: |
| https://drive.google.com/file/d/abc123xyz/view?usp=sharing |
|
|
| Next steps: |
| 1. Open the Google Drive link in your browser |
| 2. Download the ZIP file |
| 3. Extract and verify generated documents |
| ``` |
| |
| ### Test Options |
| |
| ```bash |
| # Custom number of documents |
| python test_async_api.py --google-token TOKEN --num-solutions 5 |
|
|
| # Custom API URL (if deployed) |
| python test_async_api.py --google-token TOKEN --base-url https://api.yourdomain.com |
|
|
| # Different user ID |
| python test_async_api.py --google-token TOKEN --user-id 42 |
|
|
| # With refresh token |
| python test_async_api.py \ |
| --google-token ACCESS_TOKEN \ |
| --google-refresh-token REFRESH_TOKEN |
|
|
| # Show help for getting token |
| python test_async_api.py --help-token |
| ``` |
| |
| ---Testing Sync PDF API |
| |
| The sync PDF API (`/generate/pdf`) returns results immediately (20-60s) and supports three modes of operation. Perfect for smaller batch sizes and real-time workflows. |
| |
| ### Three Operating Modes |
| |
| **Mode 1: Quick Demo (No Tracking)** |
| - Returns ZIP immediately |
| - No Supabase records created |
| - Perfect for quick testing and demos |
| - No user_id required |
| |
| **Mode 2: Demo with Tracking** |
| - Returns ZIP immediately |
| - Creates Supabase record for tracking |
| - Can poll status during generation |
| - Requires user_id |
| |
| **Mode 3: Full Production** |
| - Returns ZIP immediately |
| - Creates Supabase record |
| - Uploads to Google Drive in background |
| - Requires user_id + google_drive_token |
| - Best for production use |
| |
| ### Full Automated Test |
| |
| ```bash |
| cd docgenie/api |
|
|
| # Mode 1: Quick demo (no tracking) |
| python test_sync_pdf_api.py |
| |
| # Mode 2: Demo with tracking |
| python test_sync_pdf_api.py --user-id 123 |
|
|
| # Mode 3: Full production (tracking + GDrive) |
| python test_sync_pdf_api.py \ |
| --user-id 123 \ |
| --google-token "ya29.a0AfH6SMBx..." \ |
| --google-refresh-token "1//0gw..." |
| ``` |
| |
| **Test Flow for All Modes:** |
| 1. β Health check |
| 2. β Test Mode 1: Quick demo (always runs) |
| 3. β Test Mode 2: With tracking (if user_id provided) |
| 4. β Test Mode 3: Full production (if user_id + token provided) |
| 5. β Validate ZIP contents |
| 6. β Test status polling (Modes 2 & 3) |
| 7. β Verify GDrive upload (Mode 3) |
| |
| **Expected Output:** |
| ``` |
| ================================================================================ |
| DocGenie /generate/pdf Endpoint Test Suite |
| ================================================================================ |
| |
| ================================================================================ |
| 1. Testing API Health |
| ================================================================================ |
| β API is healthy: {'status': 'healthy', 'version': '1.0.0'} |
| |
| ================================================================================ |
| 2. Testing Mode 1: Quick Demo (No Tracking) |
| ================================================================================ |
| This mode returns ZIP immediately without creating Supabase records. |
| Use for quick testing and demos. |
| |
| Payload: |
| Seed Images: 1 |
| Num Solutions: 1 |
| User ID: None (no tracking) |
| Google Token: None |
| |
| β³ Calling /generate/pdf (expect 20-60 seconds)... |
| |
| β Response received in 42.3 seconds |
| |
| Response Headers: |
| Content-Type: application/zip |
| Content-Disposition: attachment; filename=docgenie_documents.zip |
| X-Request-ID: NOT SET (expected in mode 1) |
| X-Status-URL: NOT SET (expected in mode 1) |
|
|
| β ZIP file size: 145.2 KB |
| β ZIP contains 18 files: |
| - README.md |
| - metadata.json |
| - analysis/document_1.json |
| - annotations/gt/document_1.json |
| - bbox/bbox_pdf/word/document_1.json |
| - html/document_1.css |
| - html/document_1.html |
| - img/document_1.png |
| - pdf/pdf_final/document_1.pdf |
| - pdf/pdf_initial/document_1.pdf |
| β Contains metadata.json |
| β Contains README.md |
| |
| β
Mode 1 (Quick Demo) Test PASSED |
| β‘ Fast response: 42.3s |
| π¦ Valid ZIP file |
| β No tracking overhead |
| |
| ================================================================================ |
| 3. Testing Mode 2: Demo with Progress Tracking |
| ================================================================================ |
| This mode returns ZIP immediately AND creates Supabase record. |
| Client can poll /jobs/{request_id}/status during generation. |
|
|
| Payload: |
| User ID: 123 (tracking enabled) |
| Seed Images: 1 |
| Num Solutions: 2 |
| Google Token: None |
|
|
| β³ Calling /generate/pdf (expect 20-60 seconds)... |
|
|
| β Response received in 58.7 seconds |
|
|
| Response Headers: |
| Content-Type: application/zip |
| β X-Request-ID: 550e8400-e29b-41d4-a716-446655440000 |
| β X-Status-URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status |
|
|
| β ZIP file size: 287.4 KB |
| β ZIP contains 32 files |
| β Found 4 PDF files |
|
|
| β³ Testing status polling endpoint... |
| β Status endpoint working: |
| Request ID: 550e8400-e29b-41d4-a716-446655440000 |
| Status: completed |
| Created: 2026-03-01T10:15:00Z |
| Updated: 2026-03-01T10:15:58Z |
| β Job marked as completed |
|
|
| β
Mode 2 (Tracking) Test PASSED |
| β‘ Fast response: 58.7s |
| π¦ Valid ZIP file |
| π Progress tracking enabled |
| β Can poll status during generation |
|
|
| ================================================================================ |
| 4. Testing Mode 3: Full Production (Tracking + GDrive Upload) |
| ================================================================================ |
| This mode returns ZIP immediately AND uploads to Google Drive in background. |
| Best for production use with full tracking and backup. |
|
|
| Payload: |
| User ID: 123 |
| Google Token: ya29.a0AfH6SMBx... |
| Google Refresh: Yes |
| Seed Images: 1 |
| Num Solutions: 1 |
|
|
| β³ Calling /generate/pdf (expect 20-60 seconds)... |
|
|
| β Response received in 45.1 seconds |
|
|
| Response Headers: |
| β X-Request-ID: 660f9511-f3ac-52e5-b827-557766551111 |
| β X-Status-URL: /jobs/660f9511-f3ac-52e5-b827-557766551111/status |
|
|
| β ZIP file size: 151.8 KB |
| β ZIP contains 18 files |
|
|
| β³ ZIP returned immediately, GDrive upload happening in background... |
| (This doesn't block the response) |
|
|
| β³ Waiting 10 seconds for background GDrive upload... |
| β Status after background upload: |
| Status: completed |
| β GDrive URL: https://drive.google.com/file/d/abc123xyz/view?usp=... |
| β Background upload completed! |
|
|
| β
Mode 3 (Full Production) Test PASSED |
| β‘ Fast response: 45.1s (GDrive doesn't block) |
| π¦ Valid ZIP file delivered immediately |
| π Progress tracking enabled |
| βοΈ Google Drive backup scheduled |
| β Production-ready configuration |
|
|
| ================================================================================ |
| TEST SUMMARY |
| ================================================================================ |
| β
health: PASSED |
| β
mode_1: PASSED |
| β
mode_2: PASSED |
| β
mode_3: PASSED |
| |
| 4/4 tests passed |
| |
| π All tests passed! |
| ================================================================================ |
| ``` |
| |
| ### Test Options |
| |
| ```bash |
| # Mode 1 only (default) |
| python test_sync_pdf_api.py |
|
|
| # Mode 2 with custom user ID |
| python test_sync_pdf_api.py --user-id 456 |
| |
| # Mode 3 with custom API URL |
| python test_sync_pdf_api.py \ |
| --base-url https://api.yourdomain.com \ |
| --user-id 123 \ |
| --google-token TOKEN \ |
| --google-refresh-token REFRESH_TOKEN |
| ``` |
| |
| ### Comparing Sync vs Async |
| |
| | Feature | Sync (`/generate/pdf`) | Async (`/generate/async`) | |
| |---------|------------------------|---------------------------| |
| | **Response Time** | 20-60 seconds | 5-30 minutes | |
| | **Best For** | 1-3 documents | 5-50+ documents | |
| | **Cost** | Standard API pricing | 50% cheaper (Batch API) | |
| | **Result Delivery** | Direct ZIP download | Google Drive upload | |
| | **Progress Tracking** | Optional (Modes 2 & 3) | Always enabled | |
| | **Use Case** | Real-time workflows, demos | Bulk generation, scheduled jobs | |
| |
| **When to use Sync:** |
| - Generating 1-3 documents |
| - Need immediate results |
| - Real-time user interactions |
| - Quick testing and demos |
| |
| **When to use Async:** |
| - Generating 5+ documents |
| - Cost optimization (50% savings) |
| - Background/scheduled processing |
| - Large batch jobs |
| |
| --- |
| |
| ## Manual Testing with cURL |
| |
| ### Async API (`/generate/async`) |
| |
| #### 1. Submit Async Job |
| |
| ```bash |
| curl -X POST http://localhost:8000/generate/async \ |
| -H "Content-Type: application/json" \ |
| -d '{ |
| "user_id": 1, |
| "google_drive_token": "ya29.a0AfH6SMBx...", |
| "seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"], |
| "prompt_params": { |
| "language": "English", |
| "doc_type": "receipts", |
| "num_solutions": 2, |
| "enable_handwriting": false, |
| "enable_visual_elements": false, |
| "output_detail": "minimal" |
| } |
| }' |
| ``` |
| |
| **Response:** |
| ```json |
| { |
| "request_id": "550e8400-e29b-41d4-a716-446655440000", |
| "status": "queued", |
| "estimated_time_minutes": 10, |
| "poll_url": "/jobs/550e8400-e29b-41d4-a716-446655440000/status", |
| "created_at": "2026-02-28T12:00:00Z" |
| } |
| ``` |
|
|
| #### 2. Check Job Status |
|
|
| ```bash |
| curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status |
| ``` |
|
|
| **Response (Processing):** |
| ```json |
| { |
| "request_id": "550e8400-e29b-41d4-a716-446655440000", |
| "status": "processing", |
| "created_at": "2026-02-28T12:00:00Z", |
| "updated_at": "2026-02-28T12:02:00Z", |
| "progress": "Creating batch request..." |
| } |
| ``` |
|
|
| **Response (Completed):** |
| ```json |
| { |
| "request_id": "550e8400-e29b-41d4-a716-446655440000", |
| "status": "completed", |
| "created_at": "2026-02-28T12:00:00Z", |
| "updated_at": "2026-02-28T12:15:00Z", |
| "download_url": "https://drive.google.com/file/d/abc123xyz/view?usp=sharing", |
| "file_size_mb": 15.4, |
| "document_count": 2 |
| } |
| ``` |
|
|
| #### 3. List User Jobs |
|
|
| ```bash |
| curl "http://localhost:8000/jobs/user/1?limit=10&offset=0" |
| ``` |
|
|
| **Response:** |
| ```json |
| { |
| "user_id": 1, |
| "jobs": [ |
| { |
| "request_id": "550e8400-e29b-41d4-a716-446655440000", |
| "status": "completed", |
| "created_at": "2026-02-28T12:00:00Z", |
| "download_url": "https://drive.google.com/file/d/abc123xyz/view" |
| } |
| ], |
| "count": 1, |
| "limit": 10, |
| "offset": 0 |
| } |
| ``` |
|
|
| ### Sync PDF API (`/generate/pdf`) |
|
|
| #### Mode 1: Quick Demo (No Tracking) |
|
|
| ```bash |
| curl -X POST http://localhost:8000/generate/pdf \ |
| -H "Content-Type: application/json" \ |
| -d '{ |
| "seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"], |
| "prompt_params": { |
| "language": "English", |
| "doc_type": "receipts", |
| "num_solutions": 1, |
| "enable_handwriting": false, |
| "enable_visual_elements": false, |
| "output_detail": "minimal" |
| } |
| }' \ |
| --output documents.zip |
| ``` |
|
|
| **Response:** |
| - Returns ZIP file directly (binary) |
| - No tracking headers |
| - File saved as `documents.zip` |
|
|
| #### Mode 2: Demo with Tracking |
|
|
| ```bash |
| curl -X POST http://localhost:8000/generate/pdf \ |
| -H "Content-Type: application/json" \ |
| -d '{ |
| "user_id": 123, |
| "seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"], |
| "prompt_params": { |
| "language": "English", |
| "doc_type": "business documents", |
| "num_solutions": 2, |
| "enable_handwriting": false, |
| "output_detail": "minimal" |
| } |
| }' \ |
| --output documents.zip \ |
| -D headers.txt |
| ``` |
|
|
| **Response:** |
| - Returns ZIP file directly (binary) |
| - Headers saved to `headers.txt` contain: |
| - `X-Request-ID: 550e8400-e29b-41d4-a716-446655440000` |
| - `X-Status-URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status` |
|
|
| **Check Status:** |
| ```bash |
| # Extract request_id from headers.txt, then: |
| curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status |
| ``` |
|
|
| #### Mode 3: Full Production (Tracking + GDrive) |
|
|
| ```bash |
| curl -X POST http://localhost:8000/generate/pdf \ |
| -H "Content-Type: application/json" \ |
| -d '{ |
| "user_id": 123, |
| "google_drive_token": "ya29.a0AfH6SMBx...", |
| "google_drive_refresh_token": "1//0gw...", |
| "seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"], |
| "prompt_params": { |
| "language": "English", |
| "doc_type": "invoices", |
| "num_solutions": 1, |
| "enable_handwriting": false, |
| "output_detail": "dataset" |
| } |
| }' \ |
| --output documents.zip \ |
| -D headers.txt |
| ``` |
|
|
| **Response:** |
| - Returns ZIP file immediately (binary) |
| - Google Drive upload happens in background |
| - Wait 10-30 seconds, then check status for GDrive URL: |
|
|
| ```bash |
| curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status |
| ``` |
|
|
| **Response (after background upload):** |
| ```json |
| { |
| "request_id": "550e8400-e29b-41d4-a716-446655440000", |
| "status": "completed", |
| "created_at": "2026-03-01T10:00:00Z", |
| "updated_at": "2026-03-01T10:00:45Z", |
| "results": { |
| "download_url": "https://drive.google.com/file/d/abc123xyz/view?usp=sharing", |
| "file_size_mb": 0.15, |
| "document_count": 1, |
| "zip_filename": "docgenie_550e8400-e29b-41d4-a716-446655440000.zip" |
| } |
| } |
| ``` |
|
|
| --- |
|
|
| ## Frontend Integration Example |
|
|
| ### React + TypeScript |
|
|
| ```typescript |
| import { useState, useEffect } from 'react'; |
| |
| interface JobStatus { |
| request_id: string; |
| status: 'queued' | 'processing' | 'generating' | 'completed' | 'failed'; |
| download_url?: string; |
| error_message?: string; |
| } |
| |
| function DocumentGenerator() { |
| const [jobId, setJobId] = useState<string | null>(null); |
| const [status, setStatus] = useState<JobStatus | null>(null); |
| const [googleToken, setGoogleToken] = useState<string>(''); |
| |
| // Step 1: Google OAuth (implement separately) |
| const handleGoogleAuth = async () => { |
| // Redirect to Google OAuth |
| const clientId = 'YOUR_CLIENT_ID'; |
| const redirectUri = 'https://yourapp.com/auth/callback'; |
| const scope = 'https://www.googleapis.com/auth/drive.file'; |
| |
| const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?` + |
| `client_id=${clientId}&` + |
| `redirect_uri=${redirectUri}&` + |
| `response_type=code&` + |
| `scope=${scope}&` + |
| `access_type=offline&` + |
| `prompt=consent`; |
| |
| window.location.href = authUrl; |
| }; |
| |
| // Step 2: Submit job |
| const handleGenerateDocuments = async () => { |
| const response = await fetch('http://localhost:8000/generate/async', { |
| method: 'POST', |
| headers: { 'Content-Type': 'application/json' }, |
| body: JSON.stringify({ |
| user_id: 1, |
| google_drive_token: googleToken, |
| seed_images: ['https://example.com/seed.jpg'], |
| prompt_params: { |
| language: 'English', |
| doc_type: 'receipts', |
| num_solutions: 3 |
| } |
| }) |
| }); |
| |
| const job = await response.json(); |
| setJobId(job.request_id); |
| }; |
| |
| // Step 3: Poll status |
| useEffect(() => { |
| if (!jobId) return; |
| |
| const interval = setInterval(async () => { |
| const response = await fetch( |
| `http://localhost:8000/jobs/${jobId}/status` |
| ); |
| const data = await response.json(); |
| setStatus(data); |
| |
| if (data.status === 'completed' || data.status === 'failed') { |
| clearInterval(interval); |
| } |
| }, 30000); // Poll every 30 seconds |
| |
| return () => clearInterval(interval); |
| }, [jobId]); |
| |
| return ( |
| <div> |
| {!googleToken ? ( |
| <button onClick={handleGoogleAuth}> |
| Connect Google Drive |
| </button> |
| ) : ( |
| <button onClick={handleGenerateDocuments}> |
| Generate Documents |
| </button> |
| )} |
| |
| {status && ( |
| <div> |
| <p>Status: {status.status}</p> |
| {status.status === 'completed' && ( |
| <a href={status.download_url} target="_blank"> |
| Download Documents |
| </a> |
| )} |
| {status.status === 'failed' && ( |
| <p>Error: {status.error_message}</p> |
| )} |
| </div> |
| )} |
| </div> |
| ); |
| } |
| ``` |
|
|
| --- |
|
|
| ## Troubleshooting |
|
|
| ### Issue: "google_drive_token is required" |
|
|
| **Cause**: No token provided in request |
|
|
| **Solution**: |
| ```bash |
| # Make sure you're passing the token |
| python test_async_api.py --google-token "ya29.a0AfH6SMBx..." |
| ``` |
|
|
| ### Issue: "Failed to refresh Google Drive token" |
|
|
| **Cause**: Token expired and no refresh token provided |
|
|
| **Solutions**: |
| 1. Get a new token (tokens expire in ~1 hour) |
| 2. Include refresh token in request |
| 3. Frontend should refresh tokens automatically |
|
|
| ### Issue: "Google Drive upload failed: insufficient permissions" |
|
|
| **Cause**: Token doesn't have drive.file scope |
|
|
| **Solution**: Re-authorize with correct scope: |
| ``` |
| https://www.googleapis.com/auth/drive.file |
| ``` |
|
|
| ### Issue: Worker not processing jobs |
|
|
| **Check 1**: Is Redis running? |
| ```bash |
| redis-cli ping # Should return "PONG" |
| ``` |
|
|
| **Check 2**: Is worker running? |
| ```bash |
| # Check worker logs |
| journalctl -u docgenie-worker@1 -f |
| |
| # Or check RQ info |
| rq info --url redis://localhost:6379/0 |
| ``` |
|
|
| **Check 3**: Check failed queue |
| ```bash |
| rq info --queue failed --url redis://localhost:6379/0 |
| ``` |
|
|
| ### Issue: Job stuck in "generating" status |
|
|
| **Cause**: Batch API taking longer than expected |
|
|
| **Solution**: Wait up to 30 minutes for batched requests. Check Anthropic dashboard: |
| https://console.anthropic.com/settings/batches |
|
|
| ### Issue: Cannot access Google Drive link |
|
|
| **Cause**: File not shared properly |
|
|
| **Solution**: Check worker logs for sharing errors. File should have "anyone with link" permission. |
|
|
| --- |
|
|
| ## Performance Testing |
|
|
| ### Test Batch API Cost Savings |
|
|
| ```bash |
| # Generate 10 documents |
| time python test_async_api.py --google-token TOKEN --num-solutions 10 |
| |
| # Compare with direct API (for reference) |
| curl -X POST http://localhost:8000/generate \ |
| -H "Content-Type: application/json" \ |
| -d '{"seed_images": ["..."], "prompt_params": {"num_solutions": 10}}' |
| ``` |
|
|
| **Expected Results:** |
| - **Batched API**: 10-20 minutes, ~$2.50 per 1M tokens |
| - **Direct API**: 3-5 minutes, ~$5.00 per 1M tokens |
| - **Cost Savings**: 50% |
|
|
| --- |
|
|
| ## Next Steps |
|
|
| 1. β
Test locally with script |
| 2. β
Verify Google Drive upload |
| 3. β
Test with your frontend |
| 4. β
Deploy to production (see [DEPLOYMENT.md](DEPLOYMENT.md)) |
| 5. β
Set up monitoring (see [SCALING.md](SCALING.md)) |
|
|
| --- |
|
|
| ## Additional Resources |
|
|
| - **API Documentation**: http://localhost:8000/docs |
| - **Deployment Guide**: [DEPLOYMENT.md](DEPLOYMENT.md) |
| - **Scaling Guide**: [SCALING.md](SCALING.md) |
| - **Google OAuth Docs**: https://developers.google.com/identity/protocols/oauth2 |
| - **Anthropic Batch API**: https://docs.anthropic.com/en/docs/batch-api |
|
|