| # RTMP Integration Plan for Stack 2.9 |
|
|
| **Objective**: Replace the legacy MCP‑based tool layer with a native Python‑centric tool system while preserving existing functionality and enabling easier extension. |
|
|
| --- |
|
|
| ## 1. Architecture Overview (text diagram) |
| ``` |
| +-------------------+ +--------------------+ +-------------------+ |
| | Stack 2.9 Core | <----> | Tool Registry | <----> | Python Tool Impl | |
| | (models, prompts) | | (register/unregister) | | (functions) | |
| +-------------------+ +--------------------+ +-------------------+ |
| ^ ^ ^ |
| | | | |
| | | | |
| | +--------------------+-----------------------+ |
| | | Tool Invocation Layer (async) | |
| | | - validates input (zod -> pydantic) | |
| | | - checks permissions | |
| | | - streams progress & result | |
| | +----------------------------------------+ |
| | |
| v |
| +-------------------+ +--------------------+ +-------------------+ |
| | MCP Bridge | (legacy) | MCP Integration | (keep for backward) | |
| +-------------------+ +--------------------+ +-------------------+ |
| ``` |
| *The new system runs entirely in Python, mirroring the API surface of the existing `MCPTool` class but using native classes.* |
|
|
| --- |
|
|
| ## 2. Implementation Phases |
|
|
| ### Phase 0 – Prep & Tool Discovery |
| 1. **Add a `tools/` package** under `stack-2.9/src/` (e.g. `src/tools`). |
| 2. Create a **base `Tool` abstract class** defining: |
| - `name`, `description`, `input_schema` (pydantic), `output_schema`. |
| - `call(self, **params) -> Any` (may be async). |
| - Optional hooks: `validate`, `check_permissions`, `is_destructive`, `summary`. |
| 3. Add a **`tool_registry.py`** singleton with: |
| - `register(tool: Tool)` |
| - `unregister(name: str)` |
| - `get(name: str) -> Tool` |
| - `list() -> List[Tool]` |
| 4. Implement **auto‑loader** that scans `src/tools/**` for subclasses of `Tool` and registers them at import time (mirrors RTMP's `loadAgentsDir`). |
| 5. Write **unit tests** for registry behavior. |
| |
| ### Phase 1 – Core Tool System |
| 1. **Define schema handling** using **pydantic** (lightweight, validates JSON schemas). Provide a helper `zod_to_pydantic` if needed for future port‑over. |
| 2. Implement **`ToolInvoker`** class: |
| - `async invoke(name, params, context)` |
| - Performs input validation, permission check, runs `tool.call()`. |
| - Streams progress via async generator (`yield progress`). |
| 3. Add **permission framework** similar to RTMP's `checkPermissions`: |
| - Global policy file `tools/permissions.yaml` (allow/deny patterns). |
| - `PermissionChecker` reads the policy, supports `always_allow`, `always_deny`, `ask_user`. |
| 4. Create **example native tools** mirroring MCP built‑ins: |
| - `ReadFileTool`, `WriteFileTool`, `WebSearchTool`, `RunCommandTool`, `GitTool`. |
| - Each lives in its own module under `src/tools/`. |
| 5. Update **Stack entrypoint** (`stack-2.9/stack.py` or CLI) to import the registry and expose the new tool list to the LLM prompt (replace MCP `list_tools`). |
|
|
| ### Phase 2 – Migration Layer (MCP Compatibility) |
| 1. Keep the existing **`MCPIntegration`** class but make it a thin wrapper around the new registry: |
| - On `register_tool`, forward to `tool_registry.register`. |
| - On `call_tool`, forward to `ToolInvoker.invoke`. |
| 2. Add **adapter classes** for each legacy MCP tool that simply subclass the new `Tool` and delegate to the original implementation (e.g., `LegacyReadFileAdapter`). This ensures any existing agent definitions continue to work. |
| 3. Write **migration script** `scripts/migrate_mcp_tools.py` that: |
| - Loads all legacy MCP tools (from `RTMP/tools`). |
| - Generates stub Python classes in `src/tools/` with matching signatures. |
| - Updates `enhancements/collaboration/mcp_integration.py` to use the new registry. |
| 4. Run integration tests to confirm both legacy and native tools are discoverable. |
|
|
| ### Phase 3 – Full Cut‑over & Cleanup |
| 1. Update **prompt generation** (`agentToolUtils` etc.) to pull tool definitions from the Python registry instead of MCP metadata. |
| 2. Deprecate the MCP client in `collaboration/mcp_integration.py`: |
| - Mark as `deprecated` with warning logs. |
| - Remove automatic auto‑registration of MCP tools. |
| 3. Remove the `RTMP/tools` directory from the repo (or archive it under `legacy/`). |
| 4. Extend documentation and add **migration guide** for developers. |
| 5. bump version to `2.9.1`. |
|
|
| --- |
|
|
| ## 3. File Structure for New Tool System |
| ``` |
| stack-2.9/ |
| ├─ src/ |
| │ ├─ tools/ |
| │ │ ├─ __init__.py # registers all tools on import |
| │ │ ├─ base.py # abstract Tool class |
| │ │ ├─ registry.py # singleton registry implementation |
| │ │ ├─ invoker.py # ToolInvoker handling validation & permissions |
| │ │ ├─ permissions.yaml # policy file (allow/deny patterns) |
| │ │ ├─ read_file.py # native ReadFileTool |
| │ │ ├─ write_file.py # native WriteFileTool |
| │ │ ├─ web_search.py # native WebSearchTool (uses web_search tool) |
| │ │ ├─ run_command.py # native RunCommandTool (subprocess) |
| │ │ └─ git_tool.py # native GitTool (uses GitPython or subprocess) |
| │ └─ enhancements/ |
| │ └─ collaboration/ |
| │ ├─ __init__.py |
| │ ├─ mcp_integration.py # thin wrapper -> registry |
| │ └─ conversation_state.py |
| ├─ docs/ |
| │ └─ rtmp-integration-plan.md # <-- this file |
| └─ scripts/ |
| └─ migrate_mcp_tools.py # migration helper |
| ``` |
|
|
| --- |
|
|
| ## 4. Migration Path |
| | Step | Action | Outcome | |
| |------|--------|---------| |
| | 1️⃣ | Add native registry & base classes | Core infrastructure ready | |
| | 2️⃣ | Implement native equivalents of all built‑in MCP tools | Feature parity | |
| | 3️⃣ | Wrap legacy tools with adapters & register them via the new registry | Zero‑breakage for existing agents | |
| | 4️⃣ | Switch `list_tools` calls to `registry.list()` | Prompt now reflects native tools | |
| | 5️⃣ | Deprecate MCP client, remove old `RTMP/tools` folder | Clean codebase | |
| | 6️⃣ | Update documentation & version bump | Done | |
|
|
| --- |
|
|
| ## 5. Security & Permissions |
| - **Schema validation** via pydantic ensures only expected fields reach the implementation. |
| - **PermissionChecker** reads `tools/permissions.yaml`: |
| ```yaml |
| always_allow: |
| - read_file |
| - web_search |
| always_deny: |
| - run_command: "*" |
| ask_user: |
| - write_file |
| - git_operation |
| ``` |
| - Hooks can raise `PermissionDenied` which bubbles up to the LLM as a tool‑use error message. |
| - All destructive tools (`write_file`, `git_operation`, `run_command` with `sudo`) must be marked `is_destructive=True` to trigger explicit user confirmation. |
|
|
| --- |
|
|
| ## 6. Documentation & Testing |
| - Add **docstrings** to each tool class (auto‑generated API reference via `pydoc`). |
| - Write **pytest** suites covering: |
| - Registry CRUD |
| - Input validation failures |
| - Permission matrix (allow/deny/ask) |
| - End‑to‑end invocation through `ToolInvoker`. |
| - Update `README.md` with a *“Native Tool System”* section describing how to add new tools. |
|
|
| --- |
|
|
| ## 7. Risks & Mitigations |
| | Risk | Impact | Mitigation | |
| |------|--------|------------| |
| | Regression of existing agents relying on MCP metadata format | Breaks user workflows | Keep adapter layer until next major release; run full suite of agent integration tests. | |
| | Permission policy mis‑configuration causing silent denial | User confusion | Fail fast with clear error messages; add a startup sanity check that all tools have a permission rule. | |
| | Performance regression due to Python validation overhead | Slower turn latency | Cache compiled pydantic models; benchmark against MCP’s JSON‑schema validation (expected to be comparable). | |
|
|
| --- |
|
|
| ## 8. Next Steps |
| 1. Commit the skeleton (`src/tools/` with base classes & registry). |
| 2. Implement `ReadFileTool` and `WriteFileTool` as proof‑of‑concept. |
| 3. Run the migration script to generate adapters for the remaining MCP tools. |
| 4. Update the Stack entrypoint to expose the new tool list. |
| 5. Iterate on tests and documentation. |
|
|
| --- |
|
|
| *Prepared by the sub‑agent tasked with RTMP integration.* |
|
|
