README.md

---
title: Tokenizer Visualizer
emoji: 🔤
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: "4.44.0"
app_file: app.py
pinned: false
license: apache-2.0
tags: [tokenizer, visualization, mcp, dispatchai]
---

# 🔤 dispatchAI Tokenizer Visualizer

**MCP Server** — visualize how tokenizers split text.

## Tools
- `tokenize_text(text, method)` — Tokenize with chosen method
- `compare_methods(text)` — Compare token counts across methods

🚀 [dispatchAI](https://huggingface.co/dispatchAI)