nlproxy/docs/server.md

# NLProxy Server Module Reference

This document covers the FastAPI server implementation in the `server/` package.

## Purpose

The server module exposes HTTP APIs, handles dependency initialization, and coordinates request lifecycle management for NLProxy.

## Files and Responsibilities

### `server/config.py`

#### Purpose

Defines runtime settings using Pydantic.

#### Key Features

- Uses `ConfigDict(env_prefix="NLPROXY_", case_sensitive=False)`.
- Supports boolean parsing for env values such as `true`, `yes`, `on`, and `1`.
- Validates provider names against `LLMProvider`.

#### Configurable Values

- Server: `host`, `port`, `workers`, `log_level`
- Redis: `redis_url`, `redis_max_connections`, `redis_socket_timeout`
- Cache: `enable_semantic_cache`, `cache_similarity_threshold`, `cache_default_ttl`, `cache_embedding_dim`
- Compression: `default_aggressiveness`, `max_compression_timeout`, `compression_max_retries`
- LLM: `default_llm_provider`, `default_llm_model`, `llm_request_timeout`, `llm_max_retries`, `enable_llm_fallback`
- Safety: `min_confidence_threshold`, `max_regeneration_attempts`, `enable_auto_correction`, `privacy_mode_default`
- Observability: `enable_metrics`, `metrics_path`, `enable_tracing`, `trace_sample_rate`
- Verification: `enable_nli_verification`, `enable_perplexity_check`, `enable_semantic_drift`

### `server/main.py`

#### Purpose

Creates the FastAPI application and registers middleware, routes, exception handlers, and metrics instrumentation.

#### Key Behavior

- Uses `asynccontextmanager` for lifespan management.
- Calls `startup()` before the app becomes ready and `shutdown()` on exit.
- Registers `/docs`, `/redoc`, and `/openapi.json` endpoints.
- Conditionally instruments Prometheus metrics under `settings.metrics_path`.

### `server/dependencies.py`

#### Purpose

Initializes shared application dependencies and holds module-scoped singletons.

#### Global Objects

- `compression_service`
- `post_verifier`
- `response_corrector`
- `llm_orchestrator`
- `firewall`
- `semantic_cache`

#### Startup Flow

- Instantiates `CompressionService` with cache and model settings.
- Creates `PostLLMVerifier` and optionally hooks NLI refinement into compression.
- Creates `ResponseCorrector` and attaches it to the compression service.
- Creates `LLMOrchestrator` with fallback providers.
- Validates the default provider has credentials configured.
- Instantiates `PromptFirewall` with default regex rules and optional semantic config.
- Initializes `SemanticLLMCache` when enabled.

#### Shutdown Flow

- Calls `llm_orchestrator.close()`.
- Closes Redis connections used by cache.

### `server/middleware.py`

#### Purpose

Installs standard HTTP middleware.

#### Middleware

- `CORSMiddleware`
  - `allow_origins=["*"]`
  - `allow_methods=["*"]`
  - `allow_headers=["*"]`
- `GZipMiddleware`
  - Compresses responses larger than 1000 bytes.

#### Request ID

- Adds `X-Request-ID` header to every response.
- Stores request ID in `request.state`.

### `server/apis/chat.py`

#### Purpose

Handles chat completion requests with prompt compression, firewall analysis, LLM generation, and post-processing.

#### Key Endpoints

- `POST /v1/chat/completions`
  - Accepts `ChatCompletionRequest`
  - Returns `ChatCompletionResponse`

#### Request Flow

1. Validate service readiness.
2. Perform firewall check on user prompt.
3. Compress prompt with `CompressionService.compress_batch_async()`.
4. Apply shield and safety validation.
5. Call `LLMOrchestrator.generate()` or streaming equivalent.
6. Apply `ResponseCorrector` and `PostLLMVerifier`.
7. Optionally auto-correct low-confidence outputs.
8. Return structured response with NLProxy metadata.

#### Error Handling

- `403` for blocked prompts.
- `400` for invalid user input.
- `504` for compression or LLM timeouts.
- `502` for provider errors.
- `409` when security confidence thresholds are not met.

### `server/apis/health.py`

#### Purpose

Exposes service diagnostics and health status.

#### Endpoints

- `GET /health`
  - Returns aggregate status of compression, LLM providers, and semantic cache.
- `GET /metrics/app`
  - Returns basic request and cache metrics.

### `server/apis/errors.py`

#### Purpose

Defines global exception handlers for HTTP and internal errors.

#### Behavior

- Converts `HTTPException` into `JSONResponse` with structured error payloads.
- Handles unexpected exceptions with HTTP 500.

### `server/apis/models.py`

#### Purpose

Placeholder for model discovery APIs.

## Schemas

### `server/schemas.py`

Defines Pydantic request and response models:

- `Message`
- `ChatCompletionRequest`
- `ChatCompletionResponse`
- `HealthResponse`
- `MetricsResponse`

#### Validation Notes

- `Message.content` must not be empty.
- `ChatCompletionRequest` supports `provider`, `model`, `mode`, `aggressiveness`, `privacy_mode`, `auto_correct`, and `use_perplexity`.
- `privacy_mode` defaults to `settings.privacy_mode_default` if omitted.

### `server/logger.py`

#### Purpose

Provides FastAPI-specific logger setup.

#### Functions

- `setup_logging(level)`
- `get_request_logger(name)`

## Deployment Considerations

- The server uses an async lifespan, so startup failures should surface before binding ports.
- Metrics are only enabled if `settings.enable_metrics` is true.
- The server expects dependent modules and models to be available at startup.
- Use `uvicorn` via `cli/runserver.py` for production readiness.