Upload 3 files

# MCP server support

SERPent now speaks two protocols from the same deployment:

| Audience | Endpoint |
|---|---|
| REST clients / Swagger UI | `https://<host>/` (docs), `POST /serp/...`, `/scrap/...`, `/ops/...` |
| MCP clients (Claude, Cursor, agents) | `https://<host>/mcp` — streamable HTTP transport |

## What changed

| File | Change |
|---|---|
| `mcp_server.py` | **New.** Builds the MCP server from the FastAPI app and mounts it. |
| `app.py` | Two lines: `from mcp_server import mount_mcp_server` and `mcp = mount_mcp_server(app)` after the `include_router` calls. |
| `requirements.txt` | Added `fastmcp>=3.0,<4`. |
| `Dockerfile` | Unchanged — same image, same port 7860, same `CMD`. |

## How it works

`FastMCP.from_fastapi()` reads the app's OpenAPI schema and turns every route
into an MCP tool. When a tool is called, the request is dispatched **back into
the same FastAPI app in-process** over an ASGI transport — no network hop, no
second server, no duplicated business logic. Add an endpoint, get a tool.

The 12 tools generated today:

```
search search_arxiv search_google_scholar
search_patents search_brave search_bing
search_duck scrap_patent scrap_patents
ops_keyword_search ops_get_patent ops_get_patents_bulk
```

Tool names come from the Python handler names rather than FastAPI's generated
operation ids, so agents see `search_arxiv` instead of
`search_arxiv_serp_search_arxiv_post`. Descriptions and JSON schemas come from
your docstrings and Pydantic models — improving a docstring improves the tool.
To pin a specific tool name, set `operation_id="..."` on the route decorator.

## Connecting a client

**Claude Code / any streamable-HTTP client:**

```bash
claude mcp add --transport http serpent https://<your-space>.hf.space/mcp
```

**`.mcp.json` / `claude_desktop_config.json`:**

```json
{
"mcpServers": {
"serpent": {
"type": "http",
"url": "https://<your-space>.hf.space/mcp"
}
}
}
```

**Private HF Space** — pass your token:

```json
{
"mcpServers": {
"serpent": {
"type": "http",
"url": "https://<your-space>.hf.space/mcp",
"headers": { "Authorization": "Bearer hf_..." }
}
}
}
```

**Quick check without a client:**

```bash
curl -X POST http://localhost:7860/mcp \
-H 'Content-Type: application/json' \
-H 'Accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{
"protocolVersion":"2025-06-18","capabilities":{},
"clientInfo":{"name":"curl","version":"1"}}}'
```

Or with the FastMCP CLI: `fastmcp inspect http://localhost:7860/mcp`

## Configuration

All optional, all environment variables:

| Variable | Default | Purpose |
|---|---|---|
| `MCP_ENABLED` | `1` | Set to `0` to run REST-only. |
| `MCP_PATH` | `/mcp` | Where the MCP endpoint lives. |
| `MCP_STATELESS` | `1` | Stateless HTTP — no session affinity needed behind the HF Spaces proxy. Set `0` for resumable SSE streams. |
| `MCP_ALLOWED_HOSTS` | `*` | Comma-separated Host allow-list (DNS-rebinding protection). |
| `MCP_ALLOWED_ORIGINS` | `*` | Comma-separated Origin allow-list (browser clients). |

## Notes and caveats

- **Ordering matters.** `mount_mcp_server(app)` must come after every
`include_router()` call. Routes registered after it are not exposed as tools,
and are not reachable at all — the MCP app is mounted at the root so that a
bare `POST /mcp` answers directly instead of 307-redirecting to `/mcp/`
(a redirect some MCP clients mishandle).
- **Lifespans are combined**, so Playwright still starts up and shuts down
exactly as before, alongside the MCP session manager.
- **The `ops_*` tools** are exposed but return 503 without
`OPS_CONSUMER_KEY` / `OPS_CONSUMER_SECRET`. The server instructions tell
agents to prefer the regular tools, which already fall back to OPS.
- **Auth.** The MCP endpoint inherits whatever protects the Space. If you later
want per-token auth on MCP only, FastMCP takes an `auth=` provider in
`build_mcp_server()`.
- **Tool count.** 12 tools with six overlapping search backends is a lot of
surface for a model to choose from. If tool selection gets noisy, pass
`route_maps=[...]` to `FastMCP.from_fastapi()` in `mcp_server.py` to exclude
the per-backend tools and keep `search`, `search_arxiv`, `search_patents`,
`search_google_scholar` and the scrapers.

## Verified

Tested against `fastmcp 3.4.7` / `fastapi 0.141.1`: server boots, all 12 tools
list over HTTP with correct schemas, tool calls round-trip through the real
endpoints, and the existing REST routes (`/`, `/openapi.json`, `/redoc`,
`/serp/*`) are unaffected.

app.py

@@ -14,6 +14,7 @@ from scrap import PatentScrapBulkResponse, PatentScrapResult, scrap_patent_async
 from serp import SerpQuery, SerpResults, query_arxiv, query_bing_search, query_brave_search, query_ddg_search, query_google_patents, query_google_scholar
 from ops import OPSBulkResponse, OPSNotConfigured, ops_scrap_patent, ops_scrap_patent_bulk, ops_search, token_manager as ops_token_manager
 from utils import log_gathered_exceptions
+from mcp_server import mount_mcp_server
 
 logging.basicConfig(
     level=logging.INFO,
@@ -333,5 +334,10 @@ app.include_router(serp_router)
 app.include_router(scrap_router)
 app.include_router(ops_router)
 
+# =============================== MCP server ===================================
+# Re-exposes every endpoint above as an MCP tool over streamable HTTP at /mcp.
+# Must stay below the include_router() calls, or the routes are not picked up.
+mcp = mount_mcp_server(app)
+
 if __name__ == "__main__":
     uvicorn.run(app, host="0.0.0.0", port=7860)

mcp_server.py

@@ -0,0 +1,168 @@
+"""MCP (Model Context Protocol) server for SERPent.
+
+Every route registered on the FastAPI app is automatically re-exposed as an MCP
+tool, so a single deployment serves both audiences:
+
+    REST  ->  POST https://<host>/serp/search_arxiv
+    MCP   ->  POST https://<host>/mcp        (streamable HTTP transport)
+
+Nothing here duplicates business logic. FastMCP reads the app's OpenAPI schema
+and, when a tool is called, dispatches the request back into the same FastAPI
+app in-process (via an ASGI transport, no network round-trip), so tools always
+stay in sync with the endpoints.
+
+Usage (see the bottom of `app.py`):
+
+    from mcp_server import mount_mcp_server
+    mcp = mount_mcp_server(app)
+
+Configuration (all optional, read from the environment):
+
+    MCP_ENABLED        "0"/"false" to disable the MCP server entirely.
+    MCP_PATH           Mount path. Default "/mcp".
+    MCP_STATELESS      "0" to keep per-session state. Default stateless, which
+                       is what you want behind the HF Spaces proxy.
+    MCP_ALLOWED_HOSTS  Comma-separated Host allow-list. Default "*".
+    MCP_ALLOWED_ORIGINS Comma-separated Origin allow-list. Default "*".
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+from typing import Optional
+
+from fastapi import FastAPI
+from fastapi.routing import APIRoute
+from fastapi.utils import generate_unique_id
+
+from fastmcp import FastMCP
+from fastmcp.utilities.lifespan import combine_lifespans
+
+logger = logging.getLogger(__name__)
+
+MCP_SERVER_NAME = "SERPent"
+
+MCP_INSTRUCTIONS = """\
+SERPent gives you web, academic and patent search plus full-text patent retrieval.
+
+Typical workflow:
+  1. Search with the backend that matches the question:
+     - `search` for general web results (tries DuckDuckGo, then Brave, then Bing)
+     - `search_arxiv` for preprints, `search_google_scholar` for academic papers
+     - `search_patents` for prior art (falls back to the EPO OPS API)
+  2. Feed the patent ids you found to `scrap_patent` (one) or `scrap_patents`
+     (many, and much faster than looping) to get title, abstract, claims and
+     description.
+
+Every search tool accepts a LIST of queries and runs them concurrently, so send
+all your query variations in one call rather than making several calls. Set
+`n_results` to control how many hits come back per query.
+
+The `ops_*` tools hit the official EPO OPS API directly and only work when the
+deployment has OPS credentials configured; the regular tools already fall back
+to OPS on their own, so prefer them unless you specifically need EPO data.
+"""
+
+
+def _env_flag(name: str, default: bool) -> bool:
+    raw = os.getenv(name)
+    if raw is None:
+        return default
+    return raw.strip().lower() not in {"0", "false", "no", "off", ""}
+
+
+def _env_list(name: str, default: list[str]) -> list[str]:
+    raw = os.getenv(name)
+    if not raw:
+        return default
+    return [item.strip() for item in raw.split(",") if item.strip()]
+
+
+def _tool_names(app: FastAPI) -> dict[str, str]:
+    """Map each route's OpenAPI operationId to a clean MCP tool name.
+
+    FastAPI's generated operationIds look like
+    `search_arxiv_serp_search_arxiv_post`, which is what an LLM would otherwise
+    see. The Python handler name (`search_arxiv`) is a far better tool name, so
+    use that instead. Adding an explicit `operation_id=` to a route decorator
+    overrides this.
+    """
+    names: dict[str, str] = {}
+
+    def walk(routes) -> None:
+        for route in routes:
+            # Depending on the FastAPI version, included routers either flatten
+            # into the parent or stay nested, so recurse.
+            if isinstance(route, APIRoute):
+                generate = getattr(route, "generate_unique_id_function", None)
+                # FastAPI stores an unset callable as a DefaultPlaceholder.
+                generate = getattr(generate, "value", generate)
+                if not callable(generate):
+                    generate = generate_unique_id
+                operation_id = route.operation_id or generate(route)
+                names[operation_id] = route.name
+            # Starlette Mounts nest under `.routes`; FastAPI >= 0.141 wraps
+            # included routers in `_IncludedRouter`, which nests under
+            # `.original_router`.
+            sub = getattr(route, "routes", None)
+            if sub:
+                walk(sub)
+            included = getattr(route, "original_router", None)
+            if included is not None and getattr(included, "routes", None):
+                walk(included.routes)
+
+    walk(app.routes)
+    return names
+
+
+def build_mcp_server(app: FastAPI) -> FastMCP:
+    """Build an MCP server exposing every route of `app` as a tool."""
+    mcp = FastMCP.from_fastapi(
+        app=app,
+        name=MCP_SERVER_NAME,
+        mcp_names=_tool_names(app),
+        # Requests are dispatched straight back into this same app in-process.
+        httpx_client_kwargs={"timeout": 300.0},
+    )
+    mcp.instructions = MCP_INSTRUCTIONS
+    return mcp
+
+
+def mount_mcp_server(app: FastAPI, path: Optional[str] = None) -> Optional[FastMCP]:
+    """Mount the MCP server onto `app` (default: `/mcp`).
+
+    Must be called after every router has been included, otherwise the missing
+    routes will not show up as tools. Returns None when MCP is disabled.
+    """
+    if not _env_flag("MCP_ENABLED", True):
+        logger.info("MCP server disabled (MCP_ENABLED=0).")
+        return None
+
+    path = path or os.getenv("MCP_PATH", "/mcp")
+    path = "/" + path.strip("/")
+
+    mcp = build_mcp_server(app)
+
+    mcp_app = mcp.http_app(
+        path=path,
+        stateless_http=_env_flag("MCP_STATELESS", True),
+        allowed_hosts=_env_list("MCP_ALLOWED_HOSTS", ["*"]),
+        allowed_origins=_env_list("MCP_ALLOWED_ORIGINS", ["*"]),
+    )
+
+    # The MCP session manager needs its own lifespan to run. The app already has
+    # one (Playwright), so run both.
+    app.router.lifespan_context = combine_lifespans(
+        app.router.lifespan_context,
+        mcp_app.lifespan,
+    )
+
+    # The MCP endpoint is `path` *inside* mcp_app, and mcp_app is mounted at the
+    # root. Mounting it at `path` instead would make Starlette answer a bare
+    # `POST /mcp` with a 307 to `/mcp/`, which several MCP clients choke on.
+    # This mount is added last, so all the API routes above still match first.
+    app.mount("/", mcp_app)
+
+    logger.info("MCP server mounted at %s (streamable HTTP).", path)
+    return mcp

requirements.txt

@@ -6,4 +6,5 @@ duckduckgo_search
 beautifulsoup4
 httpx
 lxml
-python-dotenv
+python-dotenv
+fastmcp>=3.0,<4