ihtesham0345 commited on
Commit
bb2afa9
Β·
1 Parent(s): c31af39

Update docs to reflect Qwen-0.5B Local CPU architecture

Browse files
Files changed (1) hide show
  1. PROJECT_DOCS.md +24 -30
PROJECT_DOCS.md CHANGED
@@ -7,10 +7,10 @@ This project is an **AI-Powered Microservice** built with **FastAPI**. It serves
7
  - Competition Analysis
8
  - Strategic Tips
9
 
10
- **Key Feature:** This project leverages the **Qwen2.5-7B-Instruct** model, a state-of-the-art open-source LLM. Instead of running a heavy model locally, it connects to **Hugging Face's Serverless Inference API** to provide:
11
- - **High Intelligence**: Comparable to GPT-3.5/4 for reasoning.
12
- - **Speed**: Massive GPU acceleration without needing a local GPU.
13
- - **Reliability**: Always-on availability.
14
 
15
  ---
16
 
@@ -33,12 +33,12 @@ Here is how the project files are organized:
33
  ```
34
  SEO_Analyzer_FastAPI/
35
  β”œβ”€β”€ main.py # 🚦 Entry Point: Defines the API routes & server.
36
- β”œβ”€β”€ requirements.txt # πŸ“¦ Dependencies: Lists libraries (huggingface_hub, fastapi, etc.).
37
  β”œβ”€β”€ Dockerfile # 🐳 Deployment: Instructions to build the Linux container.
38
  β”œβ”€β”€ models/
39
  β”‚ └── schemas.py # πŸ“ Data Models: Pydantic classes to validate input/output.
40
  └── services/
41
- └── analyzer.py # 🧠 The Brain: Loads the AI model and handles inference.
42
  ```
43
 
44
  ---
@@ -51,23 +51,21 @@ Before writing code, we defined what the "Input" and "Output" should look like u
51
  - **Output**: A strict JSON schema ensuring the UI always receives `core_keywords`, `hashtags`, `relevance` scores, etc.
52
 
53
  ### Step 2: Building the Logic Core (`services/analyzer.py`)
54
- This is the most complex part. We use the **Hugging Face Inference Client**:
55
- 1. **Connection**: We initialize `InferenceClient(model="Qwen/Qwen2.5-7B-Instruct")`. This connects roughly to a cluster of A100 GPUs.
56
- 2. **Prompt Engineering (The "Big Brain")**: We don't just ask "give me keywords". We send a **System Prompt** that instructs the AI:
57
- > *"You are a World-Class SEO Strategist & Data Scientist... Analyze User Intent... Estimate metrics..."*
58
- This ensures the output is professional-grade, not generic.
59
- 3. **Chat Completion**: We use the `chat_completion` API format (`messages=[{"role": "system"...}, {"role": "user"...}]`) which Qwen understands perfectly.
60
- 4. **Parsing**: The model outputs a JSON string. We parse this into a Python dictionary.
61
 
62
  ### Step 3: Creating the API Endpoints (`main.py`)
63
  We created a FastAPI app with two routes:
64
  - `GET /`: A health check.
65
- - `POST /analyze-seo`: The main worker. It has robust error handling (`try/except`) to catch any API issues and report them clearly.
66
 
67
  ### Step 4: Dockerization (`Dockerfile`)
68
  To make this run on the cloud:
69
  - **Base Image**: `python:3.9`
70
- - **Optimization**: We removed heavy libraries like `torch` and `transformers`. The container is now very small and fast because the AI runs remotely.
71
  - **Port**: Exposes port **7860** for Hugging Face Spaces.
72
 
73
  ---
@@ -75,14 +73,13 @@ To make this run on the cloud:
75
  ## 5. How It Works (The Flow)
76
 
77
  1. **User Action**: Sends a request: `POST {"content": "dropshipping"}`.
78
- 2. **API Layer**: FastAPI validates the request.
79
- 3. **Business Logic**:
80
- - The `analyzer` constructs the detailed "Strategic Prompt".
81
- - It sends this prompt securely to Hugging Face's API.
82
- - **Qwen-7B** (running on powerful cloud GPUs) analyzes the text.
83
- - It returns a structured JSON response.
84
- 4. **Processing**: Python validates the fields.
85
- 5. **Response**: The user receives the data with keyword volumes, competition, and strategy.
86
 
87
  ---
88
 
@@ -93,16 +90,13 @@ To make this run on the cloud:
93
  pip install -r requirements.txt
94
  ```
95
 
96
- 2. **Set Environment Variables**:
97
- Create a `.env` file and add your token:
98
- ```
99
- HF_TOKEN=hf_...
100
- ```
101
 
102
  3. **Run the Server**:
103
  ```bash
104
  python -m uvicorn main:app --reload
105
  ```
 
106
 
107
  4. **Access Documentation**:
108
  Open `http://localhost:8000/docs`.
@@ -110,8 +104,8 @@ To make this run on the cloud:
110
  ---
111
 
112
  ## 7. Configuration Limitations
113
- - **Rate Limits**: The free Serverless API has rate limits. If you spam it, you might get a 429 error. Adding your `HF_TOKEN` increases these limits.
114
- - **Model Choice**: We selected `Qwen2.5-7B-Instruct` because it is currently the best performing 7B model available on the free tier.
115
 
116
  ---
117
  **Developed by Ihtesham | Powered by Open Source AI**
 
7
  - Competition Analysis
8
  - Strategic Tips
9
 
10
+ **Key Feature:** This project runs the **Qwen2.5-0.5B-Instruct** model **LOCALLY** inside the container.
11
+ - **Zero External Dependencies**: It does NOT use an external API. The brain lives inside the app.
12
+ - **100% Free**: No rate limits, no credit usage.
13
+ - **Privacy**: Data never leaves your container.
14
 
15
  ---
16
 
 
33
  ```
34
  SEO_Analyzer_FastAPI/
35
  β”œβ”€β”€ main.py # 🚦 Entry Point: Defines the API routes & server.
36
+ β”œβ”€β”€ requirements.txt # πŸ“¦ Dependencies: Lists libraries (torch, transformers, fastapi).
37
  β”œβ”€β”€ Dockerfile # 🐳 Deployment: Instructions to build the Linux container.
38
  β”œβ”€β”€ models/
39
  β”‚ └── schemas.py # πŸ“ Data Models: Pydantic classes to validate input/output.
40
  └── services/
41
+ └── analyzer.py # 🧠 The Brain: Loads the Local Model and handles inference.
42
  ```
43
 
44
  ---
 
51
  - **Output**: A strict JSON schema ensuring the UI always receives `core_keywords`, `hashtags`, `relevance` scores, etc.
52
 
53
  ### Step 2: Building the Logic Core (`services/analyzer.py`)
54
+ This is the heart of the "Local AI" engine:
55
+ 1. **Loading**: On startup, we use `transformers.pipeline` to download `Qwen2.5-0.5B` (approx 1GB).
56
+ 2. **Inference**: When a request comes in, the **CPU** runs the mathematical calculations to generate text.
57
+ 3. **Optimization**: We use `torch_dtype=bfloat16` to make it run faster and use less RAM.
58
+ 4. **Temperature Control**: We set `temperature=0.3` to make the AI strict and reliable for JSON.
 
 
59
 
60
  ### Step 3: Creating the API Endpoints (`main.py`)
61
  We created a FastAPI app with two routes:
62
  - `GET /`: A health check.
63
+ - `POST /analyze-seo`: The main worker. It includes a **Safety Net** that auto-fills missing data if the AI makes a mistake.
64
 
65
  ### Step 4: Dockerization (`Dockerfile`)
66
  To make this run on the cloud:
67
  - **Base Image**: `python:3.9`
68
+ - **Dependency**: We install `torch` (PyTorch) so the AI can run mathematically.
69
  - **Port**: Exposes port **7860** for Hugging Face Spaces.
70
 
71
  ---
 
73
  ## 5. How It Works (The Flow)
74
 
75
  1. **User Action**: Sends a request: `POST {"content": "dropshipping"}`.
76
+ 2. **API Layer**: FastAPI receives it.
77
+ 3. **Local Inference**:
78
+ - The server passes the text to the loaded Qwen model.
79
+ - The **CPU** generates the response token-by-token.
80
+ - This takes ~10-20 seconds.
81
+ 4. **Parsing & Repair**: The app cleans the JSON and fixes any syntax errors automatically.
82
+ 5. **Response**: The user receives the data.
 
83
 
84
  ---
85
 
 
90
  pip install -r requirements.txt
91
  ```
92
 
93
+ 2. **No Keys Needed**: You do NOT need an API key. It runs locally.
 
 
 
 
94
 
95
  3. **Run the Server**:
96
  ```bash
97
  python -m uvicorn main:app --reload
98
  ```
99
+ *Note: The first run will download the model (1GB).*
100
 
101
  4. **Access Documentation**:
102
  Open `http://localhost:8000/docs`.
 
104
  ---
105
 
106
  ## 7. Configuration Limitations
107
+ - **CPU Speed**: Since it runs on a free CPU, we limit generation to **30 keywords** to ensure it finishes quickly.
108
+ - **Model Choice**: We used the **0.5B (Nano)** model because it is the only modern LLM that fits comfortably in the free tier RAM while remaining fast.
109
 
110
  ---
111
  **Developed by Ihtesham | Powered by Open Source AI**