nlproxy/docs/core.md

NLProxy Core Module Reference

This document surveys the core library modules in core/.

Purpose

Core modules implement the prompt compression pipeline, constraint enforcement, and verification logic used by NLProxy.

Files and Responsibilities

core/compressor.py

Purpose

Performs clustering-based sentence compression to reduce prompt size while preserving core meaning.

Primary Class

• SemanticCompressor

Key Concepts

• Uses sentence embeddings to identify semantic redundancy.
• Employs KMeans clustering with n_init="auto" and max_iter=300.
• Returns compressed sentence subsets and compression metrics.

Performance

• Complexity: O(n · k · d) for clustering, where n = sentence count, k = cluster count, d = embedding dimension.
• Best suited to sentence sizes under 100 to avoid quadratic cluster overhead.

Edge Cases

• Handles empty sentence lists as a no-op.
• Falls back to conservative compression when cluster quality is low.

core/corrector.py

Purpose

Sanitizes and post-corrects LLM outputs to align with safety constraints extracted during prompt shielding.

Primary Class

• ResponseCorrector

Behavior

• Applies heuristic cleanup to generated text.
• Mends broken placeholders and ensures protected entity tokens are restored correctly.

Edge Cases

• Responds defensively when correction data is missing.
• Avoids overcorrection by preserving valid syntax.

core/model_manager.py

Purpose

Coordinates local model verification, SHA256 checksum validation, and on-demand download triggers.

Primary Classes

• ModelConfig
• ModelManager

Features

• Thread-safe singleton initialization.
• verify_zip_checksum(zip_path, expected_hash) validates download integrity.
• ensure_ready() is async-safe and idempotent.
• Supports synchronous initialization via sync_ensure_ready().

Edge Cases

• Raises RuntimeError if required models remain missing after download.
• Uses atomic file moves to avoid partial ZIP writes.

core/reconstructor.py

Purpose

Reconstructs compressed prompts by reinserting protected entities, formatting, and token optimization artifacts.

Primary Classes

• ReconstructionResult
• PromptReconstructor

Behavior

• Rebuilds final prompt text from compressed sentences and placeholder maps.
• Produces token counts and compression metrics.

Complexity

• Reconstruction is O(n) in the number of sentences and placeholder replacements.

core/restriction.py

Purpose

Represents extracted restrictions, blocklists, and prompt constraints during shielding.

Primary Classes

• Restriction
• RestrictionGraph

Behavior

• Encodes restriction rules as immutable dataclasses.
• Builds a directed graph of dependents for conflict detection.

Edge Cases

• Handles conflicting restrictions with explicit priority logic.

core/safety.py

Purpose

Validates compressed prompts and generated responses against safety policies.

Primary Classes

• SafetyReport
• SafetyChecker

Features

• Enforces semantic drift thresholds.
• Optional perplexity-based checks.
• Supports multiple safety modes.

Performance

• Safety validation is lightweight compared to embedding inference.
• Perplexity checks are conditional and only activate when enabled.

core/segmenter.py

Purpose

Segments text into sentences and generates dense embeddings.

Primary Classes

• EmbeddingBackend
• SegmentationConfig
• SemanticSegmenter

Features

• Supports ONNX and PyTorch backends.
• Uses local model artifacts from nlproxy/models/.
• Selects CPU INT8 ONNX models when onnx_int8=True.
• Exposes async inference via segment_and_encode_async().

Performance

• Sentence segmentation is O(n) in prompt length.
• Embedding inference latency depends on backend and hardware.
• Recommended batch sizes: 32-128 for CPU production.

Edge Cases

• Fails fast when local model files are missing.
• Uses NLPROXY_MODELS_DIR environment variable for model path override.

core/shield.py

Purpose

Protects sensitive content, extracts protected entities, and applies domain-specific shielding.

Primary Classes

• DomainMode
• ProtectedBlock
• ProtectedEntity
• ShieldResult
• PromptShield

Behavior

• Detects password-like tokens, email addresses, and other protected spans.
• Replaces protected text with deterministic placeholders.
• Produces ShieldResult with placeholder_map and shielded_text.

Edge Cases

• Ensures placeholder collision resistance using secrets and hashing.
• Maintains alignment for re-injection during reconstruction.

core/verifier.py

Purpose

Performs post-LLM verification and hallucination detection.

Primary Classes

• VerificationResult
• PostLLMVerifier

Behavior

• Checks final response confidence scores.
• Reports violations and corrective action recommendations.
• Integrates with automatic correction loops.

Edge Cases

• Gracefully disables NLI verification if models are unavailable.
• Allows response acceptance only when confidence_score exceeds configured thresholds.