| --- |
| license: mit |
| title: Data Extractor Using Gemini |
| sdk: docker |
| emoji: 🏃 |
| colorFrom: yellow |
| colorTo: blue |
| --- |
| # 📊 Financial Data Extractor Using Gemini |
|
|
| A powerful AI-driven financial document analysis system that automatically extracts, organizes, and generates professional Excel reports from financial documents using Google's Gemini AI models. |
|
|
| ## 🚀 Features |
|
|
| ### Core Functionality |
|
|
| - **📄 Multi-format Document Support**: PDF, DOCX, TXT, and image files |
| - **🔍 Intelligent Data Extraction**: AI-powered extraction of financial data points |
| - **📊 Smart Data Organization**: Automatic categorization into 12+ financial categories |
| - **💻 Excel Report Generation**: Professional multi-worksheet Excel reports with charts |
| - **🎯 Real-time Processing**: Live streaming interface with progress tracking |
|
|
| ### Advanced Capabilities |
|
|
| - **🤖 Multi-Agent Workflow**: Specialized AI agents for extraction, arrangement, and code generation |
| - **💾 Session Management**: Persistent storage with SQLite caching |
| - **🔄 Auto-shutdown**: Intelligent resource management for cloud deployments |
| - **📱 Modern UI**: Beautiful Gradio-based web interface |
| - **🌐 Cross-platform**: Works on Windows, Mac, and Linux |
| - **🐳 Docker Support**: Containerized deployment ready |
|
|
| ## 🏗️ Architecture |
|
|
| The system uses a sophisticated multi-agent workflow powered by the Agno framework: |
|
|
| ``` |
| 📄 Document Upload |
| ↓ |
| 🔍 Data Extractor Agent |
| ↓ (Structured Financial Data) |
| 📊 Data Arranger Agent |
| ↓ (Organized Categories) |
| 💻 Code Generator Agent |
| ↓ (Python Excel Code) |
| 📊 Excel Report Output |
| ``` |
|
|
| ### Agent Specialization |
|
|
| - **Data Extractor**: Extracts financial data points with confidence scoring |
| - **Data Arranger**: Organizes data into 12+ professional categories |
| - **Code Generator**: Creates Python code for Excel report generation |
|
|
| ## 📋 Requirements |
|
|
| ### System Requirements |
|
|
| - Python 3.8+ |
| - Google API Key (for Gemini models) |
| - 2GB+ RAM recommended |
| - Cross-platform compatible |
|
|
| ### Dependencies |
|
|
| ``` |
| agno>=1.7.4 |
| gradio |
| google-generativeai |
| PyPDF2 |
| Pillow |
| python-dotenv |
| pandas |
| matplotlib |
| openpyxl |
| python-docx |
| lxml |
| markdown |
| requests |
| seaborn |
| sqlalchemy |
| websockets |
| ``` |
|
|
| ## 🚀 Quick Start |
|
|
| ### 1. Clone the Repository |
|
|
| ```bash |
| git clone <repository-url> |
| cd Data_Extractor_Using_Gemini |
| ``` |
|
|
| ### 2. Install Dependencies |
|
|
| ```bash |
| pip install -r requirements.txt |
| ``` |
|
|
| ### 3. Configure Environment |
|
|
| Create a `.env` file: |
|
|
| ```env |
| GOOGLE_API_KEY=your_gemini_api_key_here |
| ``` |
|
|
| ### 4. Run the Application |
|
|
| ```bash |
| python app.py |
| ``` |
|
|
| The application will be available at `http://localhost:7860` |
|
|
| ## 🐳 Docker Deployment |
|
|
| ### Build and Run |
|
|
| ```bash |
| docker build -t financial-extractor . |
| docker run -p 7860:7860 --env-file .env financial-extractor |
| ``` |
|
|
| ### Environment Variables |
|
|
| - `GOOGLE_API_KEY`: Your Google Gemini API key |
| - `INACTIVITY_TIMEOUT_MINUTES`: Auto-shutdown timeout (default: 30) |
|
|
| ## 📖 Usage Guide |
|
|
| ### 1. Upload Document |
|
|
| - Drag and drop or select your financial document |
| - Supported formats: PDF, DOCX, TXT, PNG, JPG, JPEG |
|
|
| ### 2. Select Processing Mode |
|
|
| - **Quick Analysis**: Standard extraction and organization |
| - **Custom Prompts**: Use predefined prompt templates for specific document types |
|
|
| ### 3. Monitor Progress |
|
|
| - Real-time streaming interface shows each processing step |
| - Progress indicators for all workflow stages |
| - Live terminal output for code execution |
|
|
| ### 4. Download Results |
|
|
| - Professional Excel report with multiple worksheets |
| - Organized data categories with charts and formatting |
| - All intermediate files available for download |
|
|
| ## 📊 Output Structure |
|
|
| The generated Excel reports include: |
|
|
| ### Worksheets |
|
|
| - **Summary**: Executive overview with key metrics |
| - **Revenue**: Income and revenue streams |
| - **Expenses**: Operating and non-operating expenses |
| - **Assets**: Current and non-current assets |
| - **Liabilities**: Short-term and long-term liabilities |
| - **Equity**: Shareholder equity components |
| - **Cash Flow**: Cash flow statements |
| - **Ratios**: Financial ratio analysis |
| - **Charts**: Visual representations of key data |
| - **Raw Data**: Original extracted data points |
|
|
| ### Features |
|
|
| - Professional formatting with consistent styling |
| - Interactive charts and visualizations |
| - Dynamic period handling (auto-detects years/quarters) |
| - Cross-referenced data validation |
| - Print-ready layouts |
|
|
| ## 🔧 Configuration |
|
|
| ### Model Settings |
|
|
| Configure AI models in `config/settings.py`: |
|
|
| - Data Extractor Model |
| - Data Arranger Model |
| - Code Generator Model |
| - Thinking budgets and retry settings |
|
|
| ### Prompt Customization |
|
|
| Customize agent instructions in `instructions/agents/`: |
|
|
| - `data_extractor.md`: Data extraction prompts |
| - `data_arranger.md`: Data organization prompts |
| - `code_generator.md`: Excel generation prompts |
|
|
| ### Workflow Configuration |
|
|
| Modify workflow behavior in `workflow/financial_workflow.py`: |
|
|
| - Agent configurations |
| - Tool assignments |
| - Output formats |
|
|
| ## 🛠️ Development |
|
|
| ### Project Structure |
|
|
| ``` |
| ├── app.py # Main Gradio application |
| ├── workflow/ # Core workflow implementation |
| ├── instructions/ # Agent instruction templates |
| ├── prompts/ # Prompt gallery configurations |
| ├── config/ # Application settings |
| ├── utils/ # Utility functions |
| ├── static/ # Static assets |
| ├── models/ # Data models |
| └── terminal_stream.py # Real-time terminal streaming |
| ``` |
|
|
| ### Key Components |
|
|
| - **WorkflowUI**: Main interface controller |
| - **FinancialDocumentWorkflow**: Core processing pipeline |
| - **AutoShutdownManager**: Resource management |
| - **TerminalLogHandler**: Real-time logging |
| - **PromptGallery**: Template management |
|
|
| ## 🔒 Security & Privacy |
|
|
| - **Local Processing**: All document processing happens locally |
| - **No Data Storage**: Documents are processed and cleaned up automatically |
| - **API Key Security**: Environment-based configuration |
| - **Session Isolation**: Each session has isolated temporary directories |
|
|
| ## 🌐 Deployment Options |
|
|
| ### Local Development |
|
|
| ```bash |
| python app.py |
| ``` |
|
|
| ### Production (Gunicorn) |
|
|
| ```bash |
| gunicorn -w 4 -b 0.0.0.0:7860 app:app |
| ``` |
|
|
| ### Cloud Platforms |
|
|
| - **Hugging Face Spaces**: Ready for deployment |
| - **Google Cloud Run**: Containerized deployment |
| - **AWS/Azure**: Standard container deployment |
|
|
| ## 🤝 Contributing |
|
|
| 1. Fork the repository |
| 2. Create a feature branch |
| 3. Make your changes |
| 4. Add tests if applicable |
| 5. Submit a pull request |
|
|
| ## 📝 License |
|
|
| This project is licensed under the MIT License - see the LICENSE file for details. |
|
|
| ## 🆘 Support |
|
|
| ### Common Issues |
|
|
| - **API Key Errors**: Ensure your Google API key is valid and has Gemini access |
| - **Memory Issues**: Increase system RAM or reduce document size |
| - **Processing Timeouts**: Check network connectivity and API quotas |
