Updated model card
Browse files
README.md
CHANGED
|
@@ -1,3 +1,52 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
datasets:
|
| 4 |
+
- SupraLabs/chat-titles-filtered-115K
|
| 5 |
+
- ogrnz/chat-titles
|
| 6 |
+
- Michionlion/chat-titles-english
|
| 7 |
+
language:
|
| 8 |
+
- en
|
| 9 |
---
|
| 10 |
+
# TinyTitle
|
| 11 |
+
|
| 12 |
+
A tiny model (~1.8M params) that turns a chat message into a short title (2-10 words). English only. The whole thing (model + tokenizer + runtime) runs in under 5 MiB of ram, in a few tens of ms, on one small C binary.
|
| 13 |
+
|
| 14 |
+
It's a small GRU with a copy trick: for each title word it either makes up a word from its 8k vocabulary or copies a whole word straight from your message (so it keeps your spelling and casing).
|
| 15 |
+
|
| 16 |
+
Honest note: an LLM wrote almost all of this while I nodded along. It works tho. It's a toy, not a real llm.
|
| 17 |
+
|
| 18 |
+
## quickstart
|
| 19 |
+
|
| 20 |
+
This is a custom format (`.ttm1`) with a custom C runtime, so it does not work with transformers or llama.cpp. You need the runtime source from the [github repo](https://github.com/azomDev/TinyTitle).
|
| 21 |
+
|
| 22 |
+
```bash
|
| 23 |
+
# build the runtime (needs cc, nothing else)
|
| 24 |
+
cc -std=c11 -O3 -DNDEBUG -o title-v1 runtime/main.c -lm
|
| 25 |
+
|
| 26 |
+
# run it
|
| 27 |
+
./title-v1 model.ttm1 "Why does my wifi keep dropping?"
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
Files in this repo:
|
| 31 |
+
|
| 32 |
+
- `model.ttm1` - the int8 model (1.98 MB)
|
| 33 |
+
- `tokenizer-8k.json` + `tok-8k.ttok` - the tokenizer
|
| 34 |
+
- `runtime/` - main.c, ttm.h, Makefile (the whole runtime is just those files)
|
| 35 |
+
|
| 36 |
+
## numbers
|
| 37 |
+
|
| 38 |
+
| metric | value |
|
| 39 |
+
|---|---|
|
| 40 |
+
| params | ~1.8M (int8) |
|
| 41 |
+
| model file | 1.98 MB |
|
| 42 |
+
| peak rss | 4.89 mib (all pages touched) |
|
| 43 |
+
| typical cpu | ~25 ms |
|
| 44 |
+
| vocab | 8k unigram |
|
| 45 |
+
|
| 46 |
+
## train
|
| 47 |
+
|
| 48 |
+
Training needs python + torch. Full instructions in the [github repo](https://github.com/azomDev/TinyTitle). It trains in under 0.5 GB of vram.
|
| 49 |
+
|
| 50 |
+
## license
|
| 51 |
+
|
| 52 |
+
MIT. Training data comes from three public huggingface datasets ([SupraLabs/chat-titles-filtered-115K](https://huggingface.co/datasets/SupraLabs/chat-titles-filtered-115K) cc-by-4.0, [ogrnz/chat-titles](https://huggingface.co/datasets/ogrnz/chat-titles) MIT, [Michionlion/chat-titles-english](https://huggingface.co/datasets/Michionlion/chat-titles-english) cc-by-4.0), assembled by the github repo's `build_dataset.py`.
|