Testing Guide: DocGenie API
Complete guide for testing the document generation API endpoints with Google Drive integration.
Table of Contents
- Prerequisites
- Quick Start
- Getting Google Drive Token
- Testing Async API
- Testing Sync PDF API
- Manual Testing with cURL
- Frontend Integration Example
- Troubleshooting
Prerequisites
1. Start Required Services
# Terminal 1: Start Redis
## Option A: Local Redis (Recommended for Development)
# Install Redis (Ubuntu/Debian)
sudo apt-get update && sudo apt-get install redis-server -y
sudo systemctl start redis-server
sudo systemctl enable redis-server
# Verify Redis is running
redis-cli ping # Should return "PONG"
## Option B: Docker (if Docker is installed)
# docker run -d -p 6379:6379 --name redis redis:7-alpine
# Terminal 2: Start FastAPI Server
cd docgenie/api
python main.py
# Terminal 3: Start RQ Worker
cd docgenie/api
./start_worker.sh
2. Configure Environment
Make sure your api/.env file has:
# Required
ANTHROPIC_API_KEY=your_claude_api_key
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_KEY=your_supabase_key
REDIS_URL=redis://localhost:6379/0
# Optional (for token refresh)
GOOGLE_CLIENT_ID=your_client_id.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=your_client_secret
3. Create Supabase Tables
Run the SQL from DEPLOYMENT.md in your Supabase SQL Editor.
Quick Start
Option 1: Using Test Script (Easiest)
# Get Google Drive token first (one-time setup)
python api/test_get_google_token.py \
--client-id YOUR_CLIENT_ID \
--client-secret YOUR_CLIENT_SECRET
# Copy the access token, then run test
python api/test_async_api.py --google-token YOUR_ACCESS_TOKEN
Option 2: Using OAuth Playground (Quick Test)
- Go to OAuth Playground
- Configure with your credentials
- Get access token
- Run test script with token
Getting Google Drive Token
Method 1: Using Helper Script (Recommended)
Our helper script automates the OAuth flow:
cd docgenie/api
python test_get_google_token.py \
--client-id YOUR_GOOGLE_CLIENT_ID \
--client-secret YOUR_GOOGLE_CLIENT_SECRET
What it does:
- Opens browser for Google authorization
- Starts local server on port 8080 for callback
- Exchanges authorization code for tokens
- Displays access token and refresh token
Output:
Access Token: ya29.a0AfH6SMBx...
Refresh Token: 1//0gw...
Method 2: OAuth Playground (No Code)
Configure Credentials:
- Click gear icon (β) in top right
- Check "Use your own OAuth credentials"
- Enter your Client ID and Client Secret
Authorize API:
- In left panel, scroll to "Drive API v3"
- Select:
https://www.googleapis.com/auth/drive.file - Click "Authorize APIs"
- Sign in with your Google account
Get Token:
- Click "Exchange authorization code for tokens"
- Copy the "Access token" value
Method 3: Manual cURL (For Advanced Users)
Step 1: Get Authorization Code
Open this URL in browser (replace YOUR_CLIENT_ID):
https://accounts.google.com/o/oauth2/v2/auth?client_id=YOUR_CLIENT_ID&redirect_uri=http://localhost:8080&response_type=code&scope=https://www.googleapis.com/auth/drive.file&access_type=offline&prompt=consent
Step 2: Exchange Code for Token
After authorization, you'll be redirected to:
http://localhost:8080/?code=AUTHORIZATION_CODE
Exchange the code:
curl -X POST https://oauth2.googleapis.com/token \
-d "code=AUTHORIZATION_CODE" \
-d "client_id=YOUR_CLIENT_ID" \
-d "client_secret=YOUR_CLIENT_SECRET" \
-d "redirect_uri=http://localhost:8080" \
-d "grant_type=authorization_code"
Response:
{
"access_token": "ya29.a0AfH6SMBx...",
"refresh_token": "1//0gw...",
"expires_in": 3600,
"token_type": "Bearer"
}
Testing Async API
The async API (/generate/async) is optimized for batch processing with 50% cost savings. Jobs are queued and processed in the background, with status polling.
Full Automated Test
cd docgenie/api
# Set token as environment variable
export GOOGLE_DRIVE_TOKEN="ya29.a0AfH6SMBx..."
# Run test (generates 2 documents by default)
python test_async_api.py
# Or pass token directly
python test_async_api.py --google-token "ya29.a0AfH6SMBx..."
Test Flow:
- β Health check
- β Submit async job
- β Poll status (every 30 seconds)
- β List user jobs
- β Display Google Drive link
Expected Output: ```
ASYNC API TEST SUITE
================================================================================ Base URL: http://localhost:8000 User ID: 1 Documents to Generate: 2
============================================================ 1. Testing API Health
β API is healthy: {'status': 'healthy', 'version': '1.0.0'}
============================================================ 2. Submitting Async Job
Payload: User ID: 1 Seed Images: 1 Num Solutions: 2 Google Token: ya29.a0AfH6SMBx...
β Job submitted successfully! Request ID: 550e8400-e29b-41d4-a716-446655440000 Status: queued Estimated Time: 10 minutes Poll URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status
============================================================ 3. Polling Job Status
Polling every 30 seconds (max 60 attempts) Status flow: queued β processing β generating β completed/failed
[12:00:00] Poll 1/60: QUEUED [12:00:30] Poll 2/60: PROCESSING - Creating batch request... [12:01:00] Poll 3/60: GENERATING - Batch submitted to Claude... [12:08:30] Poll 17/60: GENERATING - Polling batch status... [12:15:00] Poll 30/60: COMPLETED
============================================================ β JOB COMPLETED!
Download URL: https://drive.google.com/file/d/abc123xyz/view?usp=sharing File Size: 15.4 MB Document Count: 2 Created: 2026-02-28T12:00:00Z Completed: 2026-02-28T12:15:00Z
================================================================================ TEST SUMMARY
β ALL TESTS PASSED!
Your documents are available at: https://drive.google.com/file/d/abc123xyz/view?usp=sharing
Next steps:
- Open the Google Drive link in your browser
- Download the ZIP file
- Extract and verify generated documents
### Test Options
```bash
# Custom number of documents
python test_async_api.py --google-token TOKEN --num-solutions 5
# Custom API URL (if deployed)
python test_async_api.py --google-token TOKEN --base-url https://api.yourdomain.com
# Different user ID
python test_async_api.py --google-token TOKEN --user-id 42
# With refresh token
python test_async_api.py \
--google-token ACCESS_TOKEN \
--google-refresh-token REFRESH_TOKEN
# Show help for getting token
python test_async_api.py --help-token
---Testing Sync PDF API
The sync PDF API (/generate/pdf) returns results immediately (20-60s) and supports three modes of operation. Perfect for smaller batch sizes and real-time workflows.
Three Operating Modes
Mode 1: Quick Demo (No Tracking)
- Returns ZIP immediately
- No Supabase records created
- Perfect for quick testing and demos
- No user_id required
Mode 2: Demo with Tracking
- Returns ZIP immediately
- Creates Supabase record for tracking
- Can poll status during generation
- Requires user_id
Mode 3: Full Production
- Returns ZIP immediately
- Creates Supabase record
- Uploads to Google Drive in background
- Requires user_id + google_drive_token
- Best for production use
Full Automated Test
cd docgenie/api
# Mode 1: Quick demo (no tracking)
python test_sync_pdf_api.py
# Mode 2: Demo with tracking
python test_sync_pdf_api.py --user-id 123
# Mode 3: Full production (tracking + GDrive)
python test_sync_pdf_api.py \
--user-id 123 \
--google-token "ya29.a0AfH6SMBx..." \
--google-refresh-token "1//0gw..."
Test Flow for All Modes:
- β Health check
- β Test Mode 1: Quick demo (always runs)
- β Test Mode 2: With tracking (if user_id provided)
- β Test Mode 3: Full production (if user_id + token provided)
- β Validate ZIP contents
- β Test status polling (Modes 2 & 3)
- β Verify GDrive upload (Mode 3)
Expected Output: ```
DocGenie /generate/pdf Endpoint Test Suite
================================================================================ 1. Testing API Health
β API is healthy: {'status': 'healthy', 'version': '1.0.0'}
================================================================================ 2. Testing Mode 1: Quick Demo (No Tracking)
This mode returns ZIP immediately without creating Supabase records. Use for quick testing and demos.
Payload: Seed Images: 1 Num Solutions: 1 User ID: None (no tracking) Google Token: None
β³ Calling /generate/pdf (expect 20-60 seconds)...
β Response received in 42.3 seconds
Response Headers: Content-Type: application/zip Content-Disposition: attachment; filename=docgenie_documents.zip X-Request-ID: NOT SET (expected in mode 1) X-Status-URL: NOT SET (expected in mode 1)
β ZIP file size: 145.2 KB β ZIP contains 18 files: - README.md - metadata.json - analysis/document_1.json - annotations/gt/document_1.json - bbox/bbox_pdf/word/document_1.json - html/document_1.css - html/document_1.html - img/document_1.png - pdf/pdf_final/document_1.pdf - pdf/pdf_initial/document_1.pdf β Contains metadata.json β Contains README.md
β Mode 1 (Quick Demo) Test PASSED β‘ Fast response: 42.3s π¦ Valid ZIP file β No tracking overhead
================================================================================ 3. Testing Mode 2: Demo with Progress Tracking
This mode returns ZIP immediately AND creates Supabase record. Client can poll /jobs/{request_id}/status during generation.
Payload: User ID: 123 (tracking enabled) Seed Images: 1 Num Solutions: 2 Google Token: None
β³ Calling /generate/pdf (expect 20-60 seconds)...
β Response received in 58.7 seconds
Response Headers: Content-Type: application/zip β X-Request-ID: 550e8400-e29b-41d4-a716-446655440000 β X-Status-URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status
β ZIP file size: 287.4 KB β ZIP contains 32 files β Found 4 PDF files
β³ Testing status polling endpoint... β Status endpoint working: Request ID: 550e8400-e29b-41d4-a716-446655440000 Status: completed Created: 2026-03-01T10:15:00Z Updated: 2026-03-01T10:15:58Z β Job marked as completed
β Mode 2 (Tracking) Test PASSED β‘ Fast response: 58.7s π¦ Valid ZIP file π Progress tracking enabled β Can poll status during generation
================================================================================ 4. Testing Mode 3: Full Production (Tracking + GDrive Upload)
This mode returns ZIP immediately AND uploads to Google Drive in background. Best for production use with full tracking and backup.
Payload: User ID: 123 Google Token: ya29.a0AfH6SMBx... Google Refresh: Yes Seed Images: 1 Num Solutions: 1
β³ Calling /generate/pdf (expect 20-60 seconds)...
β Response received in 45.1 seconds
Response Headers: β X-Request-ID: 660f9511-f3ac-52e5-b827-557766551111 β X-Status-URL: /jobs/660f9511-f3ac-52e5-b827-557766551111/status
β ZIP file size: 151.8 KB β ZIP contains 18 files
β³ ZIP returned immediately, GDrive upload happening in background... (This doesn't block the response)
β³ Waiting 10 seconds for background GDrive upload... β Status after background upload: Status: completed β GDrive URL: https://drive.google.com/file/d/abc123xyz/view?usp=... β Background upload completed!
β Mode 3 (Full Production) Test PASSED β‘ Fast response: 45.1s (GDrive doesn't block) π¦ Valid ZIP file delivered immediately π Progress tracking enabled βοΈ Google Drive backup scheduled β Production-ready configuration
================================================================================ TEST SUMMARY
β health: PASSED β mode_1: PASSED β mode_2: PASSED β mode_3: PASSED
4/4 tests passed
π All tests passed!
### Test Options
```bash
# Mode 1 only (default)
python test_sync_pdf_api.py
# Mode 2 with custom user ID
python test_sync_pdf_api.py --user-id 456
# Mode 3 with custom API URL
python test_sync_pdf_api.py \
--base-url https://api.yourdomain.com \
--user-id 123 \
--google-token TOKEN \
--google-refresh-token REFRESH_TOKEN
Comparing Sync vs Async
| Feature | Sync (/generate/pdf) |
Async (/generate/async) |
|---|---|---|
| Response Time | 20-60 seconds | 5-30 minutes |
| Best For | 1-3 documents | 5-50+ documents |
| Cost | Standard API pricing | 50% cheaper (Batch API) |
| Result Delivery | Direct ZIP download | Google Drive upload |
| Progress Tracking | Optional (Modes 2 & 3) | Always enabled |
| Use Case | Real-time workflows, demos | Bulk generation, scheduled jobs |
When to use Sync:
- Generating 1-3 documents
- Need immediate results
- Real-time user interactions
- Quick testing and demos
When to use Async:
- Generating 5+ documents
- Cost optimization (50% savings)
- Background/scheduled processing
- Large batch jobs
Manual Testing with cURL
Async API (/generate/async)
1. Submit Async Job
curl -X POST http://localhost:8000/generate/async \
-H "Content-Type: application/json" \
-d '{
"user_id": 1,
"google_drive_token": "ya29.a0AfH6SMBx...",
"seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"],
"prompt_params": {
"language": "English",
"doc_type": "receipts",
"num_solutions": 2,
"enable_handwriting": false,
"enable_visual_elements": false,
"output_detail": "minimal"
}
}'
Response:
{
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "queued",
"estimated_time_minutes": 10,
"poll_url": "/jobs/550e8400-e29b-41d4-a716-446655440000/status",
"created_at": "2026-02-28T12:00:00Z"
}
2. Check Job Status
curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status
Response (Processing):
{
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "processing",
"created_at": "2026-02-28T12:00:00Z",
"updated_at": "2026-02-28T12:02:00Z",
"progress": "Creating batch request..."
}
Response (Completed):
{
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "completed",
"created_at": "2026-02-28T12:00:00Z",
"updated_at": "2026-02-28T12:15:00Z",
"download_url": "https://drive.google.com/file/d/abc123xyz/view?usp=sharing",
"file_size_mb": 15.4,
"document_count": 2
}
3. List User Jobs
curl "http://localhost:8000/jobs/user/1?limit=10&offset=0"
Response:
{
"user_id": 1,
"jobs": [
{
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "completed",
"created_at": "2026-02-28T12:00:00Z",
"download_url": "https://drive.google.com/file/d/abc123xyz/view"
}
],
"count": 1,
"limit": 10,
"offset": 0
}
Sync PDF API (/generate/pdf)
Mode 1: Quick Demo (No Tracking)
curl -X POST http://localhost:8000/generate/pdf \
-H "Content-Type: application/json" \
-d '{
"seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"],
"prompt_params": {
"language": "English",
"doc_type": "receipts",
"num_solutions": 1,
"enable_handwriting": false,
"enable_visual_elements": false,
"output_detail": "minimal"
}
}' \
--output documents.zip
Response:
- Returns ZIP file directly (binary)
- No tracking headers
- File saved as
documents.zip
Mode 2: Demo with Tracking
curl -X POST http://localhost:8000/generate/pdf \
-H "Content-Type: application/json" \
-d '{
"user_id": 123,
"seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"],
"prompt_params": {
"language": "English",
"doc_type": "business documents",
"num_solutions": 2,
"enable_handwriting": false,
"output_detail": "minimal"
}
}' \
--output documents.zip \
-D headers.txt
Response:
- Returns ZIP file directly (binary)
- Headers saved to
headers.txtcontain:X-Request-ID: 550e8400-e29b-41d4-a716-446655440000X-Status-URL: /jobs/550e8400-e29b-41d4-a716-446655440000/status
Check Status:
# Extract request_id from headers.txt, then:
curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status
Mode 3: Full Production (Tracking + GDrive)
curl -X POST http://localhost:8000/generate/pdf \
-H "Content-Type: application/json" \
-d '{
"user_id": 123,
"google_drive_token": "ya29.a0AfH6SMBx...",
"google_drive_refresh_token": "1//0gw...",
"seed_images": ["https://ocr.space/Content/Images/receipt-ocr-original.webp"],
"prompt_params": {
"language": "English",
"doc_type": "invoices",
"num_solutions": 1,
"enable_handwriting": false,
"output_detail": "dataset"
}
}' \
--output documents.zip \
-D headers.txt
Response:
- Returns ZIP file immediately (binary)
- Google Drive upload happens in background
- Wait 10-30 seconds, then check status for GDrive URL:
curl http://localhost:8000/jobs/550e8400-e29b-41d4-a716-446655440000/status
Response (after background upload):
{
"request_id": "550e8400-e29b-41d4-a716-446655440000",
"status": "completed",
"created_at": "2026-03-01T10:00:00Z",
"updated_at": "2026-03-01T10:00:45Z",
"results": {
"download_url": "https://drive.google.com/file/d/abc123xyz/view?usp=sharing",
"file_size_mb": 0.15,
"document_count": 1,
"zip_filename": "docgenie_550e8400-e29b-41d4-a716-446655440000.zip"
}
}
Frontend Integration Example
React + TypeScript
import { useState, useEffect } from 'react';
interface JobStatus {
request_id: string;
status: 'queued' | 'processing' | 'generating' | 'completed' | 'failed';
download_url?: string;
error_message?: string;
}
function DocumentGenerator() {
const [jobId, setJobId] = useState<string | null>(null);
const [status, setStatus] = useState<JobStatus | null>(null);
const [googleToken, setGoogleToken] = useState<string>('');
// Step 1: Google OAuth (implement separately)
const handleGoogleAuth = async () => {
// Redirect to Google OAuth
const clientId = 'YOUR_CLIENT_ID';
const redirectUri = 'https://yourapp.com/auth/callback';
const scope = 'https://www.googleapis.com/auth/drive.file';
const authUrl = `https://accounts.google.com/o/oauth2/v2/auth?` +
`client_id=${clientId}&` +
`redirect_uri=${redirectUri}&` +
`response_type=code&` +
`scope=${scope}&` +
`access_type=offline&` +
`prompt=consent`;
window.location.href = authUrl;
};
// Step 2: Submit job
const handleGenerateDocuments = async () => {
const response = await fetch('http://localhost:8000/generate/async', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
user_id: 1,
google_drive_token: googleToken,
seed_images: ['https://example.com/seed.jpg'],
prompt_params: {
language: 'English',
doc_type: 'receipts',
num_solutions: 3
}
})
});
const job = await response.json();
setJobId(job.request_id);
};
// Step 3: Poll status
useEffect(() => {
if (!jobId) return;
const interval = setInterval(async () => {
const response = await fetch(
`http://localhost:8000/jobs/${jobId}/status`
);
const data = await response.json();
setStatus(data);
if (data.status === 'completed' || data.status === 'failed') {
clearInterval(interval);
}
}, 30000); // Poll every 30 seconds
return () => clearInterval(interval);
}, [jobId]);
return (
<div>
{!googleToken ? (
<button onClick={handleGoogleAuth}>
Connect Google Drive
</button>
) : (
<button onClick={handleGenerateDocuments}>
Generate Documents
</button>
)}
{status && (
<div>
<p>Status: {status.status}</p>
{status.status === 'completed' && (
<a href={status.download_url} target="_blank">
Download Documents
</a>
)}
{status.status === 'failed' && (
<p>Error: {status.error_message}</p>
)}
</div>
)}
</div>
);
}
Troubleshooting
Issue: "google_drive_token is required"
Cause: No token provided in request
Solution:
# Make sure you're passing the token
python test_async_api.py --google-token "ya29.a0AfH6SMBx..."
Issue: "Failed to refresh Google Drive token"
Cause: Token expired and no refresh token provided
Solutions:
- Get a new token (tokens expire in ~1 hour)
- Include refresh token in request
- Frontend should refresh tokens automatically
Issue: "Google Drive upload failed: insufficient permissions"
Cause: Token doesn't have drive.file scope
Solution: Re-authorize with correct scope:
https://www.googleapis.com/auth/drive.file
Issue: Worker not processing jobs
Check 1: Is Redis running?
redis-cli ping # Should return "PONG"
Check 2: Is worker running?
# Check worker logs
journalctl -u docgenie-worker@1 -f
# Or check RQ info
rq info --url redis://localhost:6379/0
Check 3: Check failed queue
rq info --queue failed --url redis://localhost:6379/0
Issue: Job stuck in "generating" status
Cause: Batch API taking longer than expected
Solution: Wait up to 30 minutes for batched requests. Check Anthropic dashboard: https://console.anthropic.com/settings/batches
Issue: Cannot access Google Drive link
Cause: File not shared properly
Solution: Check worker logs for sharing errors. File should have "anyone with link" permission.
Performance Testing
Test Batch API Cost Savings
# Generate 10 documents
time python test_async_api.py --google-token TOKEN --num-solutions 10
# Compare with direct API (for reference)
curl -X POST http://localhost:8000/generate \
-H "Content-Type: application/json" \
-d '{"seed_images": ["..."], "prompt_params": {"num_solutions": 10}}'
Expected Results:
- Batched API: 10-20 minutes, ~$2.50 per 1M tokens
- Direct API: 3-5 minutes, ~$5.00 per 1M tokens
- Cost Savings: 50%
Next Steps
- β Test locally with script
- β Verify Google Drive upload
- β Test with your frontend
- β Deploy to production (see DEPLOYMENT.md)
- β Set up monitoring (see SCALING.md)
Additional Resources
- API Documentation: http://localhost:8000/docs
- Deployment Guide: DEPLOYMENT.md
- Scaling Guide: SCALING.md
- Google OAuth Docs: https://developers.google.com/identity/protocols/oauth2
- Anthropic Batch API: https://docs.anthropic.com/en/docs/batch-api