Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,34 +1,47 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
emoji: π‘οΈ
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: red
|
| 6 |
sdk: docker
|
| 7 |
pinned: false
|
| 8 |
license: apache-2.0
|
| 9 |
-
short_description: '
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
-
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
```
|
| 18 |
No key β no tool β no crash β no exposed secrets
|
| 19 |
```
|
| 20 |
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
---
|
| 24 |
|
| 25 |
## Why this exists
|
| 26 |
|
| 27 |
-
|
| 28 |
|
| 29 |
-
|
| 30 |
|
| 31 |
-
This hub
|
| 32 |
|
| 33 |
- **Structural sandboxing** β `app/*` can never touch `fundaments/` or `.env`. Not by convention. By design.
|
| 34 |
- **Guardian pattern** β `main.py` is the only process that reads secrets. It injects validated services as a dict. `app/*` never sees the raw environment.
|
|
@@ -37,6 +50,47 @@ This hub was built as the antidote:
|
|
| 37 |
|
| 38 |
---
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
## Architecture
|
| 41 |
|
| 42 |
```
|
|
@@ -50,11 +104,11 @@ main.py (Guardian)
|
|
| 50 |
β
|
| 51 |
β unpacks fundaments ONCE, at startup, never stores globally
|
| 52 |
β starts hypercorn (async ASGI)
|
| 53 |
-
β routes: GET / | POST /api |
|
| 54 |
β
|
| 55 |
-
βββ app/mcp.py β FastMCP +
|
| 56 |
βββ app/tools.py β Tool registry (key-gated)
|
| 57 |
-
βββ app/
|
| 58 |
βββ app/models.py β Model limits, costs, capabilities
|
| 59 |
βββ app/config.py β .pyfun parser (single source of truth)
|
| 60 |
βββ app/db_sync.py β Internal SQLite IPC (app/* state only)
|
|
@@ -64,7 +118,7 @@ main.py (Guardian)
|
|
| 64 |
**The sandbox is structural:**
|
| 65 |
|
| 66 |
```python
|
| 67 |
-
# app/app.py β fundaments
|
| 68 |
async def start_application(fundaments: Dict[str, Any]) -> None:
|
| 69 |
config_service = fundaments["config"]
|
| 70 |
db_service = fundaments["db"] # None if not configured
|
|
@@ -80,19 +134,16 @@ This isn't documentation. It's enforced by the import structure.
|
|
| 80 |
|
| 81 |
### Why Quart + hypercorn?
|
| 82 |
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
**Quart** is async Flask β same API, same routing, but fully `async/await` native. This matters because FastMCP's SSE handler is async, and mixing sync Flask with async MCP would require thread hacks or `asyncio.run()` gymnastics. With Quart, the `/mcp` route hands off directly to `mcp.handle_sse(request)` β no bridging, no blocking.
|
| 86 |
|
| 87 |
-
**hypercorn** is an ASGI server (vs. waitress/gunicorn which are WSGI). WSGI servers handle one request per thread β
|
| 88 |
|
| 89 |
-
The `/mcp` route in `app.py`
|
| 90 |
|
| 91 |
---
|
| 92 |
|
| 93 |
## Two Databases β One Architecture
|
| 94 |
|
| 95 |
-
This hub runs **two completely separate databases** with distinct responsibilities. This is not redundancy β it's a deliberate performance and security decision.
|
| 96 |
|
| 97 |
```
|
| 98 |
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
@@ -100,14 +151,11 @@ This hub runs **two completely separate databases** with distinct responsibiliti
|
|
| 100 |
β β
|
| 101 |
β postgresql.py β Cloud DB (e.g. Neon, Supabase) β
|
| 102 |
β asyncpg pool, SSL enforced β
|
| 103 |
-
β Neon-specific quirks handled β
|
| 104 |
-
β (statement_timeout stripped, keepalives) β
|
| 105 |
β β
|
| 106 |
β user_handler.py β SQLite (users + sessions tables) β
|
| 107 |
β PBKDF2-SHA256 password hashing β
|
| 108 |
β Session validation incl. IP + UserAgent β
|
| 109 |
β Account lockout after 5 failed attempts β
|
| 110 |
-
β Path: SQLITE_PATH env var or app/ β
|
| 111 |
β β
|
| 112 |
ββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ
|
| 113 |
β inject as fundaments dict
|
|
@@ -123,12 +171,6 @@ This hub runs **two completely separate databases** with distinct responsibiliti
|
|
| 123 |
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 124 |
```
|
| 125 |
|
| 126 |
-
**Why two SQLite databases?**
|
| 127 |
-
|
| 128 |
-
`user_handler.py` (Guardian) owns `users` and `sessions` β authentication state that must be isolated from the app layer. `db_sync.py` (app/*) owns `hub_state` and `tool_cache` β fast, async IPC between tools that doesn't need to leave the process, let alone hit a cloud endpoint.
|
| 129 |
-
|
| 130 |
-
A tool caching a previous LLM response or storing intermediate state between pipeline steps should never wait on a round-trip to Neon. Local SQLite is microseconds. Cloud PostgreSQL is 50-200ms per query. For tool-to-tool communication, that difference matters.
|
| 131 |
-
|
| 132 |
**Table ownership β hard rule:**
|
| 133 |
|
| 134 |
| Table | Owner | Access |
|
|
@@ -137,20 +179,13 @@ A tool caching a previous LLM response or storing intermediate state between pip
|
|
| 137 |
| `sessions` | `fundaments/user_handler.py` | Guardian only |
|
| 138 |
| `hub_state` | `app/db_sync.py` | app/* only |
|
| 139 |
| `tool_cache` | `app/db_sync.py` | app/* only |
|
| 140 |
-
|
| 141 |
-
`db_sync.py` uses the same SQLite path (`SQLITE_PATH`) as `user_handler.py` β same file, different tables, zero overlap. The `db_query` MCP tool exposes SELECT-only access to `hub_state` and `tool_cache`. It cannot reach `users` or `sessions`.
|
| 142 |
-
|
| 143 |
-
**Cloud DB (postgresql.py):**
|
| 144 |
-
|
| 145 |
-
Handles the heavy cases β persistent storage, workflow tool results that need to survive restarts, anything that benefits from a real relational DB. Neon-specific quirks are handled automatically: `statement_timeout` is stripped from the DSN (Neon doesn't support it), SSL is enforced at `require` minimum, keepalives are set, and terminated connections trigger an automatic pool restart.
|
| 146 |
-
|
| 147 |
-
If no `DATABASE_URL` is set, the entire cloud DB layer is skipped cleanly. The app runs without it.
|
| 148 |
|
| 149 |
---
|
| 150 |
|
| 151 |
## Tools
|
| 152 |
|
| 153 |
-
Tools register
|
| 154 |
|
| 155 |
| ENV Secret | Tool | Notes |
|
| 156 |
| :--- | :--- | :--- |
|
|
@@ -160,12 +195,15 @@ Tools register themselves at startup β only if the required API key exists in
|
|
| 160 |
| `HF_TOKEN` | `llm_complete` | HuggingFace Inference API |
|
| 161 |
| `BRAVE_API_KEY` | `web_search` | Independent web index |
|
| 162 |
| `TAVILY_API_KEY` | `web_search` | AI-optimized search with synthesized answers |
|
| 163 |
-
| `DATABASE_URL` | `
|
|
|
|
| 164 |
| *(always)* | `list_active_tools` | Shows key names only β never values |
|
| 165 |
-
| *(always)* | `health_check` | Status + uptime |
|
| 166 |
| *(always)* | `get_model_info` | Limits, costs, capabilities per model |
|
| 167 |
|
| 168 |
-
|
|
|
|
|
|
|
| 169 |
|
| 170 |
```ini
|
| 171 |
[TOOL.code_review]
|
|
@@ -179,20 +217,18 @@ system_prompt = "You are an expert code reviewer. Analyze the given code for
|
|
| 179 |
```
|
| 180 |
|
| 181 |
Current built-in tools: `llm_complete`, `code_review`, `summarize`, `translate`, `web_search`, `db_query`
|
| 182 |
-
Future hooks (commented, ready): `image_gen`, `code_exec`, `
|
| 183 |
|
| 184 |
---
|
| 185 |
|
| 186 |
## LLM Fallback Chain
|
| 187 |
|
| 188 |
-
All LLM providers share one `llm_complete` tool. If a provider fails, the hub
|
| 189 |
|
| 190 |
```
|
| 191 |
-
anthropic β gemini β openrouter β huggingface
|
| 192 |
```
|
| 193 |
|
| 194 |
-
Fallbacks are configured per-provider, not hardcoded:
|
| 195 |
-
|
| 196 |
```ini
|
| 197 |
[LLM_PROVIDER.anthropic]
|
| 198 |
fallback_to = "gemini"
|
|
@@ -216,15 +252,13 @@ Same pattern applies to search providers (`brave β tavily`).
|
|
| 216 |
3. Add the API keys you have (any subset works)
|
| 217 |
4. Space starts automatically β only tools with valid keys register
|
| 218 |
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
[β Live Demo Space](https://huggingface.co/spaces/codey-lab/Universal-MCP-Hub-DEMO) (no LLM keys set!)
|
| 222 |
|
| 223 |
### Local / Docker
|
| 224 |
|
| 225 |
```bash
|
| 226 |
-
git clone https://github.com/VolkanSah/
|
| 227 |
-
cd
|
| 228 |
cp example-mcp___.env .env
|
| 229 |
# fill in your keys
|
| 230 |
pip install -r requirements.txt
|
|
@@ -240,32 +274,33 @@ LOG_TO_TMP=""
|
|
| 240 |
ENABLE_PUBLIC_LOGS="true"
|
| 241 |
HF_TOKEN=""
|
| 242 |
HUB_SPACE_URL=""
|
| 243 |
-
MCP_TRANSPORT="sse"
|
| 244 |
```
|
| 245 |
|
|
|
|
|
|
|
| 246 |
---
|
| 247 |
|
| 248 |
## Connect an MCP Client
|
| 249 |
|
| 250 |
-
###
|
| 251 |
|
| 252 |
```json
|
| 253 |
{
|
| 254 |
"mcpServers": {
|
| 255 |
"universal-mcp-hub": {
|
| 256 |
-
"url": "https://YOUR_USERNAME-universal-mcp-hub.hf.space/
|
| 257 |
}
|
| 258 |
}
|
| 259 |
}
|
| 260 |
```
|
| 261 |
|
| 262 |
-
### Private Space (with HF token)
|
| 263 |
|
| 264 |
```json
|
| 265 |
{
|
| 266 |
"mcpServers": {
|
| 267 |
"universal-mcp-hub": {
|
| 268 |
-
"url": "https://YOUR_USERNAME-universal-mcp-hub.hf.space/
|
| 269 |
"headers": {
|
| 270 |
"Authorization": "Bearer hf_..."
|
| 271 |
}
|
|
@@ -274,11 +309,29 @@ MCP_TRANSPORT="sse"
|
|
| 274 |
}
|
| 275 |
```
|
| 276 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 277 |
---
|
| 278 |
|
| 279 |
## Desktop Client
|
|
|
|
| 280 |
|
| 281 |
-
A full PySide6 desktop client is included in `DESKTOP_CLIENT/hub.py`
|
|
|
|
|
|
|
| 282 |
|
| 283 |
```bash
|
| 284 |
pip install PySide6 httpx
|
|
@@ -288,8 +341,8 @@ python DESKTOP_CLIENT/hub.py
|
|
| 288 |
```
|
| 289 |
|
| 290 |
**Features:**
|
| 291 |
-
- Multi-chat with persistent history
|
| 292 |
-
- Tool/Provider/Model selector loaded live from your Hub
|
| 293 |
- File attachments: images, PDF, CSV, Excel, ZIP, source code
|
| 294 |
- Connect tab with health check + auto-load
|
| 295 |
- Settings: HF Token + Hub URL saved locally, never sent anywhere except your own Hub
|
|
@@ -300,54 +353,60 @@ python DESKTOP_CLIENT/hub.py
|
|
| 300 |
|
| 301 |
---
|
| 302 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 303 |
## Configuration (.pyfun)
|
| 304 |
|
| 305 |
-
`app/.pyfun` is the single source of truth for all app behavior. Three tiers
|
| 306 |
|
| 307 |
```
|
| 308 |
LAZY: [HUB] + one [LLM_PROVIDER.*] β works
|
| 309 |
-
NORMAL: + [SEARCH_PROVIDER.*] + [MODELS.*] β works better
|
| 310 |
PRODUCTIVE: + [TOOLS] + [HUB_LIMITS] + [DB_SYNC] β full power
|
| 311 |
```
|
| 312 |
|
| 313 |
-
|
| 314 |
|
| 315 |
```ini
|
| 316 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 317 |
[LLM_PROVIDER.mistral]
|
| 318 |
active = "true"
|
| 319 |
base_url = "https://api.mistral.ai/v1"
|
| 320 |
env_key = "MISTRAL_API_KEY"
|
| 321 |
default_model = "mistral-large-latest"
|
| 322 |
-
models = "mistral-large-latest, mistral-small-latest
|
| 323 |
fallback_to = ""
|
| 324 |
[LLM_PROVIDER.mistral_END]
|
| 325 |
```
|
| 326 |
|
| 327 |
```python
|
| 328 |
-
# 2. app/providers.py β uncomment the dummy
|
| 329 |
_PROVIDER_CLASSES = {
|
| 330 |
...
|
| 331 |
"mistral": MistralProvider, # β uncomment to activate
|
| 332 |
}
|
| 333 |
```
|
| 334 |
|
| 335 |
-
`providers.py` ships with ready-to-use commented dummy classes for OpenAI, Mistral, and xAI/Grok β each with the matching `.pyfun` block right above it. Most OpenAI-compatible APIs need zero changes to the class itself, just a different `base_url` and `env_key`. Search providers (Brave, Tavily) follow the same pattern and are next on the roadmap.
|
| 336 |
-
|
| 337 |
-
Model limits, costs, and capabilities are also configured here β `get_model_info` reads directly from `.pyfun`:
|
| 338 |
-
|
| 339 |
-
```ini
|
| 340 |
-
[MODEL.claude-sonnet-4-6]
|
| 341 |
-
provider = "anthropic"
|
| 342 |
-
context_tokens = "200000"
|
| 343 |
-
max_output_tokens = "16000"
|
| 344 |
-
requests_per_min = "50"
|
| 345 |
-
cost_input_per_1k = "0.003"
|
| 346 |
-
cost_output_per_1k = "0.015"
|
| 347 |
-
capabilities = "text, code, analysis, vision"
|
| 348 |
-
[MODEL.claude-sonnet-4-6_END]
|
| 349 |
-
```
|
| 350 |
-
|
| 351 |
---
|
| 352 |
|
| 353 |
## Dependencies
|
|
@@ -360,20 +419,21 @@ passlib β PBKDF2 password hashing in user_handler.py
|
|
| 360 |
cryptography β encryption layer in fundaments/
|
| 361 |
|
| 362 |
# MCP Hub
|
| 363 |
-
|
| 364 |
httpx β async HTTP for all provider API calls
|
| 365 |
-
quart β async Flask (ASGI) β needed for
|
| 366 |
-
hypercorn β ASGI server β
|
| 367 |
requests β sync HTTP for tool workers
|
| 368 |
|
| 369 |
# Optional (uncomment in requirements.txt as needed)
|
| 370 |
-
# aiofiles
|
| 371 |
-
# discord.py
|
| 372 |
-
# PyNaCl
|
| 373 |
-
# psycopg2-binary
|
| 374 |
```
|
| 375 |
|
| 376 |
-
|
|
|
|
| 377 |
|
| 378 |
---
|
| 379 |
|
|
@@ -383,8 +443,9 @@ The core stack is intentionally lean. `asyncpg` + `quart` + `hypercorn` + `fastm
|
|
| 383 |
- `list_active_tools` returns key **names** only β never values
|
| 384 |
- `db_query` is SELECT-only, enforced at application level (not just docs)
|
| 385 |
- `app/*` has zero import access to `fundaments/` internals
|
| 386 |
-
- Direct execution of `app/app.py`
|
| 387 |
-
- `fundaments/`
|
|
|
|
| 388 |
|
| 389 |
> PyFundaments is not perfect. But it's more secure than most of what runs in production today.
|
| 390 |
|
|
@@ -394,25 +455,34 @@ The core stack is intentionally lean. `asyncpg` + `quart` + `hypercorn` + `fastm
|
|
| 394 |
|
| 395 |
## Foundation
|
| 396 |
|
| 397 |
-
|
| 398 |
|
| 399 |
- `config_handler.py` β env loading with validation
|
| 400 |
- `postgresql.py` β async DB pool (Guardian-only)
|
| 401 |
- `encryption.py` β key-based encryption layer
|
| 402 |
- `access_control.py` β role/permission management
|
| 403 |
-
- `user_handler.py` β user lifecycle management
|
| 404 |
- `security.py` β unified security manager composing the above
|
| 405 |
|
| 406 |
-
None
|
| 407 |
|
| 408 |
[β PyFundaments Function Overview](PyFundaments%20β%20Function%20Overview.md)
|
| 409 |
-
[β Module Docs](docs/app/)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 410 |
|
| 411 |
---
|
| 412 |
|
| 413 |
## History
|
| 414 |
|
| 415 |
-
[ShellMaster](https://github.com/VolkanSah/ChatGPT-ShellMaster) (2023, MIT) was the precursor β browser-accessible shell for ChatGPT with session memory
|
| 416 |
|
| 417 |
---
|
| 418 |
|
|
@@ -428,6 +498,6 @@ By using this software you agree to all ethical constraints defined in ESOL v1.1
|
|
| 428 |
---
|
| 429 |
|
| 430 |
*Architecture, security decisions, and PyFundaments by Volkan KΓΌcΓΌkbudak.*
|
| 431 |
-
*Built with Claude (Anthropic) as a typing assistant for docs
|
| 432 |
|
| 433 |
> crafted with passion β just wanted to understand how it works, don't actually need it, have a CLI π
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Multi-LLM API Gateway
|
| 3 |
emoji: π‘οΈ
|
| 4 |
colorFrom: indigo
|
| 5 |
colorTo: red
|
| 6 |
sdk: docker
|
| 7 |
pinned: false
|
| 8 |
license: apache-2.0
|
| 9 |
+
short_description: 'Secure Multi-LLM Gateway β (Streamable HTTP / SSE)'
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Multi-LLM API Gateway
|
| 13 |
|
| 14 |
+
β or Universal MCP Hub (Sandboxed)
|
| 15 |
+
β or secure AI wrapper with dual interface: REST + MCP
|
| 16 |
+
|
| 17 |
+
aka: a clean, secure starting point for your own projects.
|
| 18 |
+
Pick the description that fits your use case. They're all correct.
|
| 19 |
+
|
| 20 |
+
> A production-grade **the-thing** that actually thinks about security.
|
| 21 |
+
> Built on [PyFundaments](PyFundaments.md) β running on **simpleCity**.
|
| 22 |
|
| 23 |
```
|
| 24 |
No key β no tool β no crash β no exposed secrets
|
| 25 |
```
|
| 26 |
|
| 27 |
+
> [!WARNING]
|
| 28 |
+
> Most MCP servers are prompts dressed up as servers. This one has a real architecture.
|
| 29 |
+
|
| 30 |
+
---
|
| 31 |
+
|
| 32 |
+
> [!IMPORTANT]
|
| 33 |
+
> This project is under active development β always use the latest release from [Codey Lab](https://github.com/Codey-LAB/Multi-LLM-API-Gateway) *(more stable builds land here first)*.
|
| 34 |
+
> This repo ([DEV](https://github.com/VolkanSah/Multi-LLM-API-Gateway)) is where the chaos happens. π¬ A β on the repos will be cool π
|
| 35 |
|
| 36 |
---
|
| 37 |
|
| 38 |
## Why this exists
|
| 39 |
|
| 40 |
+
The AI ecosystem is full of servers with hardcoded keys, `os.environ` scattered everywhere, zero sandboxing. One misconfigured fork and your API keys are gone.
|
| 41 |
|
| 42 |
+
This is exactly the kind of negligence (and worse β outright fraud) that [Wall of Shames](https://github.com/Wall-of-Shames) documents: fake "AI tools" exploiting non-technical users β API wrappers dressed up as custom models, Telegram payment funnels, bought stars. If you build on open source, you should know this exists.
|
| 43 |
|
| 44 |
+
This hub is the antidote:
|
| 45 |
|
| 46 |
- **Structural sandboxing** β `app/*` can never touch `fundaments/` or `.env`. Not by convention. By design.
|
| 47 |
- **Guardian pattern** β `main.py` is the only process that reads secrets. It injects validated services as a dict. `app/*` never sees the raw environment.
|
|
|
|
| 50 |
|
| 51 |
---
|
| 52 |
|
| 53 |
+
## Two Interfaces β One Server
|
| 54 |
+
|
| 55 |
+
This hub exposes **two completely independent interfaces** on the same hypercorn instance:
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
POST /api β REST interface β for custom clients, desktop apps, CMS plugins
|
| 59 |
+
GET+POST /mcp β MCP interface β for Claude Desktop, Cursor, Windsurf, any MCP client
|
| 60 |
+
GET / β Health check β uptime, status
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
They share the same tool registry, provider config, and fallback chain. Adding a tool once makes it available on both interfaces automatically.
|
| 64 |
+
|
| 65 |
+
### REST API (`/api`)
|
| 66 |
+
|
| 67 |
+
Simple JSON POST β no protocol overhead, works with any HTTP client:
|
| 68 |
+
|
| 69 |
+
```json
|
| 70 |
+
POST /api
|
| 71 |
+
{"tool": "llm_complete", "params": {"prompt": "Hello", "provider": "anthropic"}}
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
Used by: Desktop Client (`DESKTOP_CLIENT/hub.py`), WordPress plugin, any custom integration.
|
| 75 |
+
|
| 76 |
+
### MCP Interface (`/mcp`)
|
| 77 |
+
|
| 78 |
+
Full MCP protocol β tool discovery, structured calls, streaming responses.
|
| 79 |
+
|
| 80 |
+
**Primary transport: Streamable HTTP** (MCP spec 2025-11-25)
|
| 81 |
+
**Fallback transport: SSE** (legacy, configurable via `.pyfun`)
|
| 82 |
+
|
| 83 |
+
Configured via `HUB_TRANSPORT` in `app/.pyfun [HUB]`:
|
| 84 |
+
|
| 85 |
+
```ini
|
| 86 |
+
HUB_TRANSPORT = "streamable-http" # default β MCP spec 2025-11-25
|
| 87 |
+
# HUB_TRANSPORT = "sse" # legacy fallback for older clients
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
Used by: Claude Desktop, Cursor, Windsurf, any MCP-compatible client.
|
| 91 |
+
|
| 92 |
+
---
|
| 93 |
+
|
| 94 |
## Architecture
|
| 95 |
|
| 96 |
```
|
|
|
|
| 104 |
β
|
| 105 |
β unpacks fundaments ONCE, at startup, never stores globally
|
| 106 |
β starts hypercorn (async ASGI)
|
| 107 |
+
β routes: GET / | POST /api | /mcp (transport-dependent)
|
| 108 |
β
|
| 109 |
+
βββ app/mcp.py β FastMCP + transport handler (Streamable HTTP / SSE)
|
| 110 |
βββ app/tools.py β Tool registry (key-gated)
|
| 111 |
+
βββ app/providers.py β LLM + Search execution + fallback chain
|
| 112 |
βββ app/models.py β Model limits, costs, capabilities
|
| 113 |
βββ app/config.py β .pyfun parser (single source of truth)
|
| 114 |
βββ app/db_sync.py β Internal SQLite IPC (app/* state only)
|
|
|
|
| 118 |
**The sandbox is structural:**
|
| 119 |
|
| 120 |
```python
|
| 121 |
+
# app/app.py β fundaments unpacked ONCE, NEVER stored globally
|
| 122 |
async def start_application(fundaments: Dict[str, Any]) -> None:
|
| 123 |
config_service = fundaments["config"]
|
| 124 |
db_service = fundaments["db"] # None if not configured
|
|
|
|
| 134 |
|
| 135 |
### Why Quart + hypercorn?
|
| 136 |
|
| 137 |
+
**Quart** is async Flask β fully `async/await` native. FastMCP's handlers are async; mixing sync Flask would require thread hacks. With Quart, `/mcp` hands off directly to FastMCP β no bridging, no blocking.
|
|
|
|
|
|
|
| 138 |
|
| 139 |
+
**hypercorn** is an ASGI server (vs. waitress/gunicorn which are WSGI). WSGI servers handle one request per thread β wrong for long-lived MCP connections. hypercorn handles both Streamable HTTP and SSE natively, and runs without extra config on HuggingFace Spaces. HTTP/2 support (`config.h2 = True`) is built-in β relevant for Streamable HTTP performance at scale.
|
| 140 |
|
| 141 |
+
The `/mcp` route in `app.py` remains the natural interception point regardless of transport β auth checks, rate limiting, and logging can all be added there before the request reaches FastMCP.
|
| 142 |
|
| 143 |
---
|
| 144 |
|
| 145 |
## Two Databases β One Architecture
|
| 146 |
|
|
|
|
| 147 |
|
| 148 |
```
|
| 149 |
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
|
|
| 151 |
β β
|
| 152 |
β postgresql.py β Cloud DB (e.g. Neon, Supabase) β
|
| 153 |
β asyncpg pool, SSL enforced β
|
|
|
|
|
|
|
| 154 |
β β
|
| 155 |
β user_handler.py β SQLite (users + sessions tables) β
|
| 156 |
β PBKDF2-SHA256 password hashing β
|
| 157 |
β Session validation incl. IP + UserAgent β
|
| 158 |
β Account lockout after 5 failed attempts β
|
|
|
|
| 159 |
β β
|
| 160 |
ββββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββ
|
| 161 |
β inject as fundaments dict
|
|
|
|
| 171 |
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 172 |
```
|
| 173 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 174 |
**Table ownership β hard rule:**
|
| 175 |
|
| 176 |
| Table | Owner | Access |
|
|
|
|
| 179 |
| `sessions` | `fundaments/user_handler.py` | Guardian only |
|
| 180 |
| `hub_state` | `app/db_sync.py` | app/* only |
|
| 181 |
| `tool_cache` | `app/db_sync.py` | app/* only |
|
| 182 |
+
| `hub_results` | PostgreSQL / Guardian | via `persist_result` tool |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 183 |
|
| 184 |
---
|
| 185 |
|
| 186 |
## Tools
|
| 187 |
|
| 188 |
+
Tools register at startup β only if the required API key exists. No key, no tool. Server always starts.
|
| 189 |
|
| 190 |
| ENV Secret | Tool | Notes |
|
| 191 |
| :--- | :--- | :--- |
|
|
|
|
| 195 |
| `HF_TOKEN` | `llm_complete` | HuggingFace Inference API |
|
| 196 |
| `BRAVE_API_KEY` | `web_search` | Independent web index |
|
| 197 |
| `TAVILY_API_KEY` | `web_search` | AI-optimized search with synthesized answers |
|
| 198 |
+
| `DATABASE_URL` | `cloud DB` | e.g. Neon, Supabase |
|
| 199 |
+
| `DATABASE_URL` | `db_query`, `persist_result` | SQLite read + PostgreSQL write |
|
| 200 |
| *(always)* | `list_active_tools` | Shows key names only β never values |
|
| 201 |
+
| *(always)* | `health_check` | Status + uptime + active transport |
|
| 202 |
| *(always)* | `get_model_info` | Limits, costs, capabilities per model |
|
| 203 |
|
| 204 |
+
For all key names see [`app/.pyfun`](app/.pyfun).
|
| 205 |
+
|
| 206 |
+
**Tools are configured in `.pyfun` β including system prompts:**
|
| 207 |
|
| 208 |
```ini
|
| 209 |
[TOOL.code_review]
|
|
|
|
| 217 |
```
|
| 218 |
|
| 219 |
Current built-in tools: `llm_complete`, `code_review`, `summarize`, `translate`, `web_search`, `db_query`
|
| 220 |
+
Future hooks (commented, ready): `image_gen`, `code_exec`, `shellmaster_2.0`, Discord, GitHub webhooks
|
| 221 |
|
| 222 |
---
|
| 223 |
|
| 224 |
## LLM Fallback Chain
|
| 225 |
|
| 226 |
+
All LLM providers share one `llm_complete` tool. If a provider fails, the hub walks the fallback chain from `.pyfun`:
|
| 227 |
|
| 228 |
```
|
| 229 |
+
e.g. anthropic β gemini β openrouter β huggingface
|
| 230 |
```
|
| 231 |
|
|
|
|
|
|
|
| 232 |
```ini
|
| 233 |
[LLM_PROVIDER.anthropic]
|
| 234 |
fallback_to = "gemini"
|
|
|
|
| 252 |
3. Add the API keys you have (any subset works)
|
| 253 |
4. Space starts automatically β only tools with valid keys register
|
| 254 |
|
| 255 |
+
[β Live Demo Space](https://huggingface.co/spaces/codey-lab/Multi-LLM-API-Gateway) (no LLM keys set)
|
|
|
|
|
|
|
| 256 |
|
| 257 |
### Local / Docker
|
| 258 |
|
| 259 |
```bash
|
| 260 |
+
git clone https://github.com/VolkanSah/Multi-LLM-API-Gateway
|
| 261 |
+
cd Multi-LLM-API-Gateway
|
| 262 |
cp example-mcp___.env .env
|
| 263 |
# fill in your keys
|
| 264 |
pip install -r requirements.txt
|
|
|
|
| 274 |
ENABLE_PUBLIC_LOGS="true"
|
| 275 |
HF_TOKEN=""
|
| 276 |
HUB_SPACE_URL=""
|
|
|
|
| 277 |
```
|
| 278 |
|
| 279 |
+
Transport is configured in `app/.pyfun [HUB]` β not via ENV.
|
| 280 |
+
|
| 281 |
---
|
| 282 |
|
| 283 |
## Connect an MCP Client
|
| 284 |
|
| 285 |
+
### Streamable HTTP (default β MCP spec 2025-11-25)
|
| 286 |
|
| 287 |
```json
|
| 288 |
{
|
| 289 |
"mcpServers": {
|
| 290 |
"universal-mcp-hub": {
|
| 291 |
+
"url": "https://YOUR_USERNAME-universal-mcp-hub.hf.space/mcp"
|
| 292 |
}
|
| 293 |
}
|
| 294 |
}
|
| 295 |
```
|
| 296 |
|
| 297 |
+
### Streamable HTTP β Private Space (with HF token)
|
| 298 |
|
| 299 |
```json
|
| 300 |
{
|
| 301 |
"mcpServers": {
|
| 302 |
"universal-mcp-hub": {
|
| 303 |
+
"url": "https://YOUR_USERNAME-universal-mcp-hub.hf.space/mcp",
|
| 304 |
"headers": {
|
| 305 |
"Authorization": "Bearer hf_..."
|
| 306 |
}
|
|
|
|
| 309 |
}
|
| 310 |
```
|
| 311 |
|
| 312 |
+
### SSE legacy fallback (set `HUB_TRANSPORT = "sse"` in `.pyfun`)
|
| 313 |
+
|
| 314 |
+
```json
|
| 315 |
+
{
|
| 316 |
+
"mcpServers": {
|
| 317 |
+
"universal-mcp-hub": {
|
| 318 |
+
"url": "https://YOUR_USERNAME-universal-mcp-hub.hf.space/mcp"
|
| 319 |
+
}
|
| 320 |
+
}
|
| 321 |
+
}
|
| 322 |
+
```
|
| 323 |
+
|
| 324 |
+
> Same URL (`/mcp`) for both transports β the protocol is negotiated automatically.
|
| 325 |
+
> SSE fallback is for older clients that don't support Streamable HTTP yet.
|
| 326 |
+
|
| 327 |
---
|
| 328 |
|
| 329 |
## Desktop Client
|
| 330 |
+
###### (experimental β ~80% AI generated)
|
| 331 |
|
| 332 |
+
A full PySide6 desktop client is included in `DESKTOP_CLIENT/hub.py`.
|
| 333 |
+
Communicates via the REST `/api` endpoint β no MCP protocol overhead.
|
| 334 |
+
Ideal for private or non-public Spaces.
|
| 335 |
|
| 336 |
```bash
|
| 337 |
pip install PySide6 httpx
|
|
|
|
| 341 |
```
|
| 342 |
|
| 343 |
**Features:**
|
| 344 |
+
- Multi-chat with persistent history
|
| 345 |
+
- Tool / Provider / Model selector loaded live from your Hub
|
| 346 |
- File attachments: images, PDF, CSV, Excel, ZIP, source code
|
| 347 |
- Connect tab with health check + auto-load
|
| 348 |
- Settings: HF Token + Hub URL saved locally, never sent anywhere except your own Hub
|
|
|
|
| 353 |
|
| 354 |
---
|
| 355 |
|
| 356 |
+
## CMS & Custom Clients
|
| 357 |
+
|
| 358 |
+
| Client | Interface used | Notes |
|
| 359 |
+
| :--- | :--- | :--- |
|
| 360 |
+
| [Desktop Client](DESKTOP_CLIENT/hub.py) | REST `/api` | PySide6, local |
|
| 361 |
+
| [WP AI Hub](https://github.com/VolkanSah/WP-AI-HUB/) | REST `/api` | WordPress plugin |
|
| 362 |
+
| TYPO3 (soon) | REST `/api` | β |
|
| 363 |
+
| Claude Desktop | MCP `/mcp` | Streamable HTTP |
|
| 364 |
+
| Cursor / Windsurf | MCP `/mcp` | Streamable HTTP |
|
| 365 |
+
|
| 366 |
+
---
|
| 367 |
+
|
| 368 |
## Configuration (.pyfun)
|
| 369 |
|
| 370 |
+
`app/.pyfun` is the single source of truth for all app behavior. Three tiers:
|
| 371 |
|
| 372 |
```
|
| 373 |
LAZY: [HUB] + one [LLM_PROVIDER.*] β works
|
| 374 |
+
NORMAL: + [SEARCH_PROVIDER.*] + [MODELS.*] β works better
|
| 375 |
PRODUCTIVE: + [TOOLS] + [HUB_LIMITS] + [DB_SYNC] β full power
|
| 376 |
```
|
| 377 |
|
| 378 |
+
Key settings in `[HUB]`:
|
| 379 |
|
| 380 |
```ini
|
| 381 |
+
[HUB]
|
| 382 |
+
HUB_TRANSPORT = "streamable-http" # streamable-http | sse
|
| 383 |
+
HUB_STATELESS = "true" # true = HF Spaces safe, no session state
|
| 384 |
+
HUB_PORT = "7860"
|
| 385 |
+
[HUB_END]
|
| 386 |
+
```
|
| 387 |
+
|
| 388 |
+
Adding a new LLM provider β two steps:
|
| 389 |
+
|
| 390 |
+
```ini
|
| 391 |
+
# 1. app/.pyfun
|
| 392 |
[LLM_PROVIDER.mistral]
|
| 393 |
active = "true"
|
| 394 |
base_url = "https://api.mistral.ai/v1"
|
| 395 |
env_key = "MISTRAL_API_KEY"
|
| 396 |
default_model = "mistral-large-latest"
|
| 397 |
+
models = "mistral-large-latest, mistral-small-latest"
|
| 398 |
fallback_to = ""
|
| 399 |
[LLM_PROVIDER.mistral_END]
|
| 400 |
```
|
| 401 |
|
| 402 |
```python
|
| 403 |
+
# 2. app/providers.py β uncomment the dummy
|
| 404 |
_PROVIDER_CLASSES = {
|
| 405 |
...
|
| 406 |
"mistral": MistralProvider, # β uncomment to activate
|
| 407 |
}
|
| 408 |
```
|
| 409 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 410 |
---
|
| 411 |
|
| 412 |
## Dependencies
|
|
|
|
| 419 |
cryptography β encryption layer in fundaments/
|
| 420 |
|
| 421 |
# MCP Hub
|
| 422 |
+
mcp β MCP protocol + FastMCP (Streamable HTTP + SSE)
|
| 423 |
httpx β async HTTP for all provider API calls
|
| 424 |
+
quart β async Flask (ASGI) β needed for MCP + hypercorn
|
| 425 |
+
hypercorn β ASGI server β Streamable HTTP + SSE, HF Spaces native
|
| 426 |
requests β sync HTTP for tool workers
|
| 427 |
|
| 428 |
# Optional (uncomment in requirements.txt as needed)
|
| 429 |
+
# aiofiles β async file ops (ML pipelines, file uploads)
|
| 430 |
+
# discord.py β Discord bot integration (planned)
|
| 431 |
+
# PyNaCl β Discord signature verification
|
| 432 |
+
# psycopg2-binary β alternative PostgreSQL driver
|
| 433 |
```
|
| 434 |
|
| 435 |
+
> **Note:** The package is `mcp` (not `fastmcp`) β `FastMCP` is imported from `mcp.server.fastmcp`.
|
| 436 |
+
> Streamable HTTP support requires `mcp >= 1.6.0`.
|
| 437 |
|
| 438 |
---
|
| 439 |
|
|
|
|
| 443 |
- `list_active_tools` returns key **names** only β never values
|
| 444 |
- `db_query` is SELECT-only, enforced at application level (not just docs)
|
| 445 |
- `app/*` has zero import access to `fundaments/` internals
|
| 446 |
+
- Direct execution of `app/app.py` blocked by design β warning + null-fundaments fallback
|
| 447 |
+
- `fundaments/` initialized conditionally β missing services degrade gracefully, never crash
|
| 448 |
+
- Streamable HTTP uses standard Bearer headers β no token-in-URL (unlike SSE)
|
| 449 |
|
| 450 |
> PyFundaments is not perfect. But it's more secure than most of what runs in production today.
|
| 451 |
|
|
|
|
| 455 |
|
| 456 |
## Foundation
|
| 457 |
|
| 458 |
+
Built on [PyFundaments](PyFundaments.md) β a security-first Python boilerplate:
|
| 459 |
|
| 460 |
- `config_handler.py` β env loading with validation
|
| 461 |
- `postgresql.py` β async DB pool (Guardian-only)
|
| 462 |
- `encryption.py` β key-based encryption layer
|
| 463 |
- `access_control.py` β role/permission management
|
| 464 |
+
- `user_handler.py` β user lifecycle management
|
| 465 |
- `security.py` β unified security manager composing the above
|
| 466 |
|
| 467 |
+
None accessible from `app/*`. Injected as a validated dict by `main.py`.
|
| 468 |
|
| 469 |
[β PyFundaments Function Overview](PyFundaments%20β%20Function%20Overview.md)
|
| 470 |
+
[β Module Docs](docs/app/)
|
| 471 |
+
[β Source Repo](https://github.com/VolkanSah/Multi-LLM-API-Gateway)
|
| 472 |
+
|
| 473 |
+
---
|
| 474 |
+
|
| 475 |
+
## Related Projects
|
| 476 |
+
|
| 477 |
+
- [Customs LLMs for free β Build Your Own LLM Service](https://github.com/VolkanSah/SmolLM2-customs/)
|
| 478 |
+
- [WP AI Hub (WordPress Client)](https://github.com/VolkanSah/WP-AI-HUB/)
|
| 479 |
+
- [ShellMaster (2023 precursor)](https://github.com/VolkanSah/ChatGPT-ShellMaster)
|
| 480 |
|
| 481 |
---
|
| 482 |
|
| 483 |
## History
|
| 484 |
|
| 485 |
+
[ShellMaster](https://github.com/VolkanSah/ChatGPT-ShellMaster) (2023, MIT) was the precursor β browser-accessible shell for ChatGPT with session memory, built before MCP was a concept. Universal MCP Hub is its natural evolution: same idea, proper architecture, dual interface.
|
| 486 |
|
| 487 |
---
|
| 488 |
|
|
|
|
| 498 |
---
|
| 499 |
|
| 500 |
*Architecture, security decisions, and PyFundaments by Volkan KΓΌcΓΌkbudak.*
|
| 501 |
+
*Built with Claude (Anthropic) as a typing assistant for docs (and the occasional bug).*
|
| 502 |
|
| 503 |
> crafted with passion β just wanted to understand how it works, don't actually need it, have a CLI π
|