Spaces:
Running
Running
File size: 3,257 Bytes
2129c29 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | # NLProxy CLI Module Reference
This document covers the command-line interface implemented under `cli/`.
## Purpose
The CLI package provides operational tooling for NLProxy:
- Starting the server
- Downloading and validating local models
- Running compression tasks from the command line
- Running project tests
## Files
### `cli/runserver.py`
#### Responsibilities
- Parses CLI arguments for server host, port, worker count, provider selection, and logging.
- Loads `.env` values from the current working directory.
- Configures provider API keys and exports them to environment variables.
- Starts `uvicorn` with production-ready settings.
#### Notable CLI Behavior
- `--reload` forces `workers=1` to avoid conflicts with auto-reload.
- Default provider is resolved from CLI flag, environment variables, or built-in fallback.
- Logs an error and exits if provider credentials are missing.
- Uses `uvicorn.run(..., http="httptools", ws="websockets")` for performance.
#### Configuration Options
- `--host`, `--port`
- `--workers`
- `--llm-client`
- `--model`
- `--api-key-client`
- `--reload`
- `--log-level`
- `--access-log`
- `--list-models`
#### Environment Variables
- `NLPROXY_HOST`
- `NLPROXY_PORT`
- `NLPROXY_LOG_LEVEL`
- `NLPROXY_DEFAULT_LLM_PROVIDER`
- `NLPROXY_DEFAULT_LLM_MODEL`
- `NLPROXY_MODELS_URL`
### `cli/download_models.py`
#### Responsibilities
- Downloads the repository model archive from a configurable URL.
- Extracts model files safely and validates required directory structure.
- Supports checksum verification for `nlproxy_models.zip`.
- Reports progress via `tqdm` when available.
#### Primary Functions
- `ensure_models_dir(models_dir: Path) -> None`
- `download_with_retries(url: str, dest_path: Path) -> bool`
- `extract_and_validate(zip_path: Path, models_dir: Path, expected_dirs: List[str]) -> bool`
- `cleanup_temp(zip_path: Path, keep_zip: bool) -> None`
#### Validation Rules
- Rejects archives containing absolute paths or `..` entries.
- Ensures expected directories exist after extraction:
- `all-MiniLM-L6-v2`
- `nli-distilroberta-base`
- `distilgpt2`
- Fails if any extracted model directory lacks valid model artifact files.
### `cli/compress.py`
#### Responsibilities
- Provides a terminal wrapper for one-off compression operations.
- Accepts prompt text, compression mode, aggressiveness, and output formatting.
- Integrates with the service layer for the same pipeline used by the server.
#### General Behavior
- Supports `--quiet` to reduce console output.
- Outputs JSON by default.
- Reuses the same compressor logic as the FastAPI route.
### `cli/help.py`
#### Responsibilities
- Prints CLI usage help text.
- Acts as a lightweight documentation helper for users who invoke the CLI with `--help`.
### `cli/tests.py`
#### Responsibilities
- Wraps `pytest` invocation for repository tests.
- Builds pytest command line based on CLI flags.
- Supports coverage and custom test selection.
## Usage Notes
- The CLI is intentionally thin and delegates heavy work to `server`, `service`, and `core` modules.
- The downloader is the only CLI entrypoint that performs local filesystem writes and network I/O.
- `runserver` is the recommended path for production deployments.
|