| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-Coder-0.5B-Instruct |
| library_name: transformers |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - bash |
| - shell |
| - cli |
| - terminal |
| - nl2bash |
| - code-generation |
| - gguf |
| - qwen2 |
| - small-language-model |
| --- |
| |
| # 🐱 Kitty Bash LLM |
|
|
| **A 0.5B parameter shell assistant that runs on a laptop CPU.** |
|
|
| Kitty Bash LLM turns plain English into bash commands, repairs commands that just |
| failed, and completes half-typed ones. It is deliberately small — the 4-bit GGUF |
| is **398 MB** and generates a command in well under a second on an ordinary CPU, |
| so it can sit behind a shell integration without a GPU, an API key, or a network |
| round trip. |
|
|
| It is a **specialist, not a chatbot.** Ask it the capital of France and you will get |
| nonsense. Ask it to find every file over 100 MB and it will tell you. |
|
|
| --- |
|
|
| ## What it does |
|
|
| Three behaviours, selected by the system prompt. One model, one file, flat memory. |
|
|
| | task | you give it | it returns | |
| |------|-------------|------------| |
| | **`nl2cmd`** | `find all files larger than 100MB under /var` | `find /var -type f -size +100M` | |
| | **`fixcmd`** | `$ gerp -r 'TODO' .`<br>`bash: gerp: command not found` | `grep -r 'TODO' .` | |
| | **`complete`** | `tar -czf backup.` | `$(date +%F).tar.gz /path/to/dir` | |
|
|
| ### System prompts |
|
|
| ```python |
| NL2CMD = ("You are a bash command generator. Given a natural language request, " |
| "output only the bash command that accomplishes it. " |
| "No explanation, no markdown fences.") |
| |
| FIXCMD = ("You fix broken bash commands. Given a failed command and its error " |
| "output, output only the corrected command. No explanation.") |
| |
| COMPLETE = ("You complete partially typed bash commands. Given a command prefix, " |
| "output only the text that completes it. No explanation.") |
| ``` |
|
|
| --- |
|
|
| ## Quick start |
|
|
| ### llama.cpp / GGUF (recommended — this is what it was built for) |
|
|
| ```bash |
| llama-cli -m kitty-bash-llm-q4_k_m.gguf -st -t 4 --no-display-prompt -n 64 --temp 0 \ |
| -p "<|im_start|>system |
| You are a bash command generator. Given a natural language request, output only the bash command that accomplishes it. No explanation, no markdown fences.<|im_end|> |
| <|im_start|>user |
| show which process is listening on port 8080<|im_end|> |
| <|im_start|>assistant |
| " |
| # -> lsof -i :8080 |
| ``` |
|
|
| For anything interactive, run a **persistent server** so the system prompt is |
| prefilled once (see *Performance* — this is the single most important |
| implementation detail): |
|
|
| ```bash |
| llama-server -m kitty-bash-llm-q4_k_m.gguf -c 1024 -t 4 --port 8080 |
| ``` |
|
|
| ### transformers |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("sahellx/kitty-bash-llm") |
| tok = AutoTokenizer.from_pretrained("sahellx/kitty-bash-llm") |
| |
| SYSTEM = ("You are a bash command generator. Given a natural language request, " |
| "output only the bash command that accomplishes it. " |
| "No explanation, no markdown fences.") |
| |
| msgs = [{"role": "system", "content": SYSTEM}, |
| {"role": "user", "content": "compress the logs folder into a tar.gz archive"}] |
| ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt") |
| out = model.generate(ids, max_new_tokens=64, do_sample=False) |
| print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)) |
| # -> tar -czvf logs.tar.gz logs/ |
| ``` |
|
|
| Greedy decoding (`do_sample=False` / `--temp 0`) is recommended. You want the |
| most likely command, not a creative one. |
|
|
| --- |
|
|
| ## Evaluation |
|
|
| Measured against the **untuned base model** on a held-out split, with identical |
| prompts, identical f16 GGUF format, and the same llama.cpp engine — the only |
| variable is the fine-tuning. 100 examples per task. |
|
|
| ### `nl2cmd` |
|
|
| | metric | base | **Kitty** | Δ | |
| |---|---|---|---| |
| | exact match | 7.00 | **23.00** | **+16.00** (3.3×) | |
| | first-utility accuracy | 61.00 | **79.00** | **+18.00** | |
| | token F1 | 39.35 | **59.77** | +20.42 | |
| | `bash -n` valid | 93.00 | **94.00** | +1.00 | |
| | ShellCheck clean | 77.00 | **82.00** | +5.00 | |
|
|
| ### `fixcmd` |
|
|
| | metric | base | **Kitty** | Δ | |
| |---|---|---|---| |
| | exact match | 0.00 | **82.00** | +82.00 | |
| | first-utility accuracy | 1.00 | **99.00** | +98.00 | |
| | token F1 | 45.20 | **96.71** | +51.51 | |
| | ShellCheck clean | 63.00 | **83.00** | +20.00 | |
|
|
| > ⚠️ **Read this number sceptically.** The base model scores ~0 largely because it |
| > *ignores the output format* and explains the error in prose rather than emitting a |
| > command — so much of the gap is format compliance, not repair skill. The test set |
| > also uses synthetic corruptions of three kinds the model trained on. On **real** |
| > shell errors, hands-on testing put it closer to **1-in-3**. Treat 82% as an |
| > upper bound. |
|
|
| ### `complete` |
|
|
| | metric | base | **Kitty** | Δ | |
| |---|---|---|---| |
| | token F1 | 2.22 | **29.88** | **+27.66** (13×) | |
| | exact match | 0.00 | 7.00 | +7.00 | |
| | prefix+completion valid | — | **99.29** | — | |
|
|
| > Exact match is a poor metric here — many completions are valid for any prefix. |
| > `bash -n` on the completion *fragment alone* is meaningless (a fragment like |
| > `--rm ubuntu` is not valid standalone bash); the meaningful figure is |
| > prefix + completion, at **99.29%**. |
|
|
| ### Comparison with other shell-focused models |
|
|
| Alongside the base-model comparison above, an **informal** benchmark was run against |
| four other publicly available fine-tunes for shell/linux command generation, all at |
| Q4_K_M (except where a repo ships only f16), through llama.cpp on the same 2-thread |
| CPU, using the same 80 held-out prompts. |
|
|
| | model | params | size | utility acc | token F1 | s/cmd | |
| |---|---|---|---|---|---| |
| | **Kitty Bash LLM** | **0.5B** | **398 MB** | **78.75** | **59.54** | **1.68** | |
| | `louisguthmann/qwen3.5-2b-shellcommand-linux` | 2.0B | 1274 MB | 63.75 | 32.02 | 15.39 | |
| | `vitali87/shell-commands-qwen2-1.5b-extended` | 1.5B | 3094 MB | 57.50 | 33.93 | 9.48 | |
| | `mecha-org/linux-command-generator-llama3.2-1b` | 1.0B | 808 MB | 52.50 | 34.21 | 2.78 | |
| | `chamibuddhika/linux-commands-0407-00` | — | 1709 MB | — | — | — | |
|
|
| > #### ⚠️ Please read this before citing the table |
| > |
| > **This is indicative, not a controlled evaluation.** Four specific caveats, all of |
| > which favour this model: |
| > |
| > 1. **The test set is drawn from this model's own training distribution.** The split |
| > is properly held out with zero leakage, but it comes from the same corpora and |
| > shares their formatting conventions. The other models never saw that style. This |
| > inflates exact match in particular, which is why exact match is omitted above. |
| > 2. **All models received *this* model's system prompt.** Peers trained with a |
| > different prompt format are being evaluated off-distribution, which alone could |
| > account for a large part of the gap. |
| > 3. **n = 80.** The 95% confidence interval is roughly ±9 points; the gap to the |
| > runner-up is significant only marginally (p ≈ 0.03). |
| > 4. **`chamibuddhika/linux-commands-0407-00` returned empty output for every prompt** |
| > and is reported as no-result rather than zero — the cause may well be the harness |
| > rather than the model. `qwen3.5-shellcommand` emits `</think>` reasoning tags that |
| > the harness did not strip, so its syntax-validity score was invalid and has been |
| > omitted here. |
| > |
| > A properly controlled comparison would use a neutral test set none of the models |
| > trained on, each model's own prompt format, and n ≥ 500. Treat the ordering above as |
| > a hint worth verifying, not a settled result. |
| > |
| > The **base-model comparison** in the previous section does not carry these caveats: |
| > same architecture, same data, same harness, only the fine-tuning differs. |
|
|
| ShellCheck is used because `bash -n` only proves a command *parses*. ShellCheck |
| catches genuine defects — unquoted expansions (`SC2086`), word splitting |
| (`SC2046`) — that parse fine and then break on a filename with a space. |
|
|
| --- |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | base | [`Qwen/Qwen2.5-Coder-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) | |
| | method | LoRA `r=64`, `alpha=64`, all attention + MLP projections, merged | |
| | precision | fp16 (Tesla T4, no bf16) | |
| | epochs / lr | 1 / `2e-4` cosine, 3% warmup | |
| | batch | 16 × 2 grad-accum = 32 effective | |
| | max seq len | 512 | |
| | loss | response-only (masked so loss lands on the command, never the prompt) | |
| | runtime | 41 min on a single T4 | |
| | final train / eval loss | 0.5579 / **0.5919** | |
|
|
| The base model was **chosen by measurement, not assumption**: a bake-off against |
| `Qwen2.5-0.5B-Instruct` under an identical recipe gave 84.98 vs 84.21 utility |
| accuracy, so the code-pretrained variant won. |
|
|
| --- |
|
|
| ## Dataset |
|
|
| ~60,000 examples across the three tasks, pooled from seven sources, deduplicated, |
| rebalanced, and decontaminated. |
|
|
| | split | nl2cmd | fixcmd | complete | total | |
| |---|---|---|---|---| |
| | train | 22,573 | 19,891 | 17,528 | **59,992** | |
| | val | 1,254 | 1,238 | 1,456 | 3,948 | |
| | test | 1,254 | 1,219 | 1,409 | 3,882 | |
|
|
| **Sources:** [`AnishJoshi/nl2bash-custom`](https://huggingface.co/datasets/AnishJoshi/nl2bash-custom), |
| [`neulab/tldr`](https://huggingface.co/datasets/neulab/tldr), |
| [`emirkaanozdemr/bash_command_data_6K`](https://huggingface.co/datasets/emirkaanozdemr/bash_command_data_6K), |
| [`huytd189/command-line-suggestions`](https://huggingface.co/datasets/huytd189/command-line-suggestions), |
| [`aelhalili/bash-commands-dataset`](https://huggingface.co/datasets/aelhalili/bash-commands-dataset), |
| [`Jawajawa/command-linux-bash-balanced-sft`](https://huggingface.co/datasets/Jawajawa/command-linux-bash-balanced-sft), |
| [`PocketDoc/Dans-Toolmaxx-ShellCommands`](https://huggingface.co/datasets/PocketDoc/Dans-Toolmaxx-ShellCommands). |
|
|
| ### Decontamination — please read this if you benchmark on nl2bash |
|
|
| The most widely used NL2Bash split on the Hub **leaks badly**. Measured directly: |
|
|
| ``` |
| test prompts also present in train : 71.0 % |
| dev prompts also present in train : 71.8 % |
| duplicate rows within train : 39.5 % (19,658 rows → 11,890 unique) |
| ``` |
|
|
| Training and evaluating on those splits measures memorisation. Every published |
| number here comes from splits rebuilt from scratch: all rows pooled, deduplicated |
| on the normalised prompt, re-split 90/5/5 with a fixed seed, and asserted in code |
| to have **zero** train↔test overlap. |
|
|
| Two further corrections: |
|
|
| - **`find` rebalanced 54% → 30%.** In the source corpus more than half of all |
| targets were `find`, which made the model a `find` specialist that produced |
| nonsense for `tar` and `du`. Capping it fixed both. |
| - **Gold targets filtered through `bash -n`.** Broken targets teach broken bash. |
| - **`fixcmd` and `complete` derived *after* the split**, from within each split |
| only — so a command in test can never appear in train under a different task label. |
|
|
| --- |
|
|
| ## Limitations |
|
|
| Findings from hands-on testing, not guesses. |
|
|
| **Weak on multi-step logic.** It picks the right tool but can get the reasoning wrong: |
|
|
| ``` |
| "print the 3rd column of a csv where the 1st column equals ERROR" |
| → cut -d',' -f3 data.csv | grep ERROR ✗ filters the wrong column |
| correct: awk -F, '$1=="ERROR" {print $3}' |
| |
| "show the 10 biggest directories under /home sorted by size" |
| → find /home -type d | sort -n -r | head -10 ✗ sorts names, not sizes |
| correct: du -h /home/* | sort -rh | head -10 |
| ``` |
|
|
| **Uneven across utilities.** Strong on `find`, `tar`, `docker`, `sed`, `grep`, `lsof`. |
| Weaker on less common subcommands (`git branch --merged` came out as gibberish). |
|
|
| **`fixcmd` is much weaker on real errors than the benchmark suggests** — see the |
| warning above. |
|
|
| **`complete` may duplicate a prefix boundary.** Given `docker run -it --` it can |
| return `--rm ...`. **Strip the overlap between prefix and completion in your client.** |
|
|
| **Catastrophic forgetting — by design.** General ability is gone. This is a bash |
| tool, not an assistant. |
|
|
| **No safety behaviour whatsoever.** Asked to delete the root filesystem it emits |
| the command immediately, with no warning. See below. |
|
|
| --- |
|
|
| ## ⚠️ Safety |
|
|
| **This model will generate destructive commands without hesitation.** |
|
|
| ``` |
| "delete every file in the root filesystem" → find / -exec rm {} \; |
| ``` |
|
|
| It has no refusal training and no notion of danger. Any tool built on it **must**: |
|
|
| 1. **Never auto-execute.** Print the command; require an explicit keypress. |
| 2. **Pattern-match destructive commands** — `rm -rf`, `dd`, `mkfs`, `> /dev/sd*`, |
| `chmod -R 777`, `curl … | sh` — and warn loudly. |
| 3. **Treat output as untrusted.** It is a suggestion from a 0.5B model, not an |
| authority. |
|
|
| Verify commands before running them. Especially ones touching `/`. |
|
|
| --- |
|
|
| ## Performance |
|
|
| Measured with llama.cpp, Q4_K_M, on **2 CPU threads** (deliberately weak): |
|
|
| | | | |
| |---|---| |
| | prompt processing | 42.8 tok/s | |
| | generation | 17.1 tok/s | |
| | 40-token system prompt prefill | 935 ms | |
| | ~12-token command generation | 701 ms | |
| | **cold request** | **1636 ms** | |
| | **warm (KV cache reused)** | **701 ms** | |
|
|
| **Implication: run a persistent server.** Over half of a cold request is spent |
| re-processing the same system prompt. Spawning a process per invocation costs |
| ~1.6 s; a warm `llama-server` costs ~700 ms. On a typical 8-core machine expect |
| roughly 3–4× faster (~200 ms). |
|
|
| For inline autosuggestion, that is still too slow to run on every keystroke. |
| **Debounce (~250 ms), fire asynchronously, and cancel in-flight requests** — the |
| Copilot pattern. History-based matching should handle the common case; use the |
| model for what history has never seen. |
|
|
| --- |
|
|
| ## Files |
|
|
| | file | size | use | |
| |---|---|---| |
| | `kitty-bash-llm-q4_k_m.gguf` | 398 MB | **recommended** — CPU inference | |
| | `kitty-bash-llm-q5_k_m.gguf` | 420 MB | slightly higher quality | |
| | `kitty-bash-llm-q8_0.gguf` | 531 MB | near-lossless | |
| | `kitty-bash-llm-f16.gguf` | 994 MB | full precision GGUF | |
| | `model.safetensors` | 988 MB | transformers / further fine-tuning | |
|
|
| --- |
|
|
| ## Reproducing this |
|
|
| Every step is a script, not a notebook cell: data pooling and decontamination, |
| the base-model bake-off, training, GGUF conversion, and the evaluation harness |
| (including the ShellCheck pass). The `find` rebalancing and the leakage assertions |
| are the parts worth copying if you build on the NL2Bash corpora. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{kitty-bash-llm, |
| title = {Kitty Bash LLM: a 0.5B shell assistant for CPU inference}, |
| author = {sahellx}, |
| year = {2026}, |
| url = {https://huggingface.co/sahellx/kitty-bash-llm} |
| } |
| ``` |
|
|
| Built on [Qwen2.5-Coder](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B-Instruct) |
| (Apache-2.0), trained with [Unsloth](https://github.com/unslothai/unsloth), |
| quantized with [llama.cpp](https://github.com/ggml-org/llama.cpp). |
|
|
