English
azomDev commited on
Commit
63cbc6e
·
verified ·
1 Parent(s): 2f02765

Updated model card

Browse files
Files changed (1) hide show
  1. README.md +49 -0
README.md CHANGED
@@ -1,3 +1,52 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - SupraLabs/chat-titles-filtered-115K
5
+ - ogrnz/chat-titles
6
+ - Michionlion/chat-titles-english
7
+ language:
8
+ - en
9
  ---
10
+ # TinyTitle
11
+
12
+ A tiny model (~1.8M params) that turns a chat message into a short title (2-10 words). English only. The whole thing (model + tokenizer + runtime) runs in under 5 MiB of ram, in a few tens of ms, on one small C binary.
13
+
14
+ It's a small GRU with a copy trick: for each title word it either makes up a word from its 8k vocabulary or copies a whole word straight from your message (so it keeps your spelling and casing).
15
+
16
+ Honest note: an LLM wrote almost all of this while I nodded along. It works tho. It's a toy, not a real llm.
17
+
18
+ ## quickstart
19
+
20
+ This is a custom format (`.ttm1`) with a custom C runtime, so it does not work with transformers or llama.cpp. You need the runtime source from the [github repo](https://github.com/azomDev/TinyTitle).
21
+
22
+ ```bash
23
+ # build the runtime (needs cc, nothing else)
24
+ cc -std=c11 -O3 -DNDEBUG -o title-v1 runtime/main.c -lm
25
+
26
+ # run it
27
+ ./title-v1 model.ttm1 "Why does my wifi keep dropping?"
28
+ ```
29
+
30
+ Files in this repo:
31
+
32
+ - `model.ttm1` - the int8 model (1.98 MB)
33
+ - `tokenizer-8k.json` + `tok-8k.ttok` - the tokenizer
34
+ - `runtime/` - main.c, ttm.h, Makefile (the whole runtime is just those files)
35
+
36
+ ## numbers
37
+
38
+ | metric | value |
39
+ |---|---|
40
+ | params | ~1.8M (int8) |
41
+ | model file | 1.98 MB |
42
+ | peak rss | 4.89 mib (all pages touched) |
43
+ | typical cpu | ~25 ms |
44
+ | vocab | 8k unigram |
45
+
46
+ ## train
47
+
48
+ Training needs python + torch. Full instructions in the [github repo](https://github.com/azomDev/TinyTitle). It trains in under 0.5 GB of vram.
49
+
50
+ ## license
51
+
52
+ MIT. Training data comes from three public huggingface datasets ([SupraLabs/chat-titles-filtered-115K](https://huggingface.co/datasets/SupraLabs/chat-titles-filtered-115K) cc-by-4.0, [ogrnz/chat-titles](https://huggingface.co/datasets/ogrnz/chat-titles) MIT, [Michionlion/chat-titles-english](https://huggingface.co/datasets/Michionlion/chat-titles-english) cc-by-4.0), assembled by the github repo's `build_dataset.py`.