Running
Agents
NeoMME Visual Memory
🔍
Search uploaded images and PDFs with natural language queries
nikki PRO
b4ph
AI & ML interests
Composer, Audio Director, Musician
Mathematician working in Digital Forensics
You Know What You Have To Do.
Recent Activity
repliedto their post about 16 hours ago
For those who wanna run Qwen3-4B on CPU alone, I tested the whole ladder:
Same machine for everything: CPU only, 16 threads, no GPU offload. Perplexity is WikiText-2 raw, which llama.cpp also uses in CI.
```text
Quant Size PPL vs F16 tg t/s
F16 8.05 GB 13.4304 — 2.52
Q8_0 4.28 GB 13.4409 +0.01 5.04
Q6_K 3.31 GB 13.4404 +0.01 5.93
Q5_K_M 2.89 GB 13.5362 +0.11 5.64
Q4_K_M 2.50 GB 13.7304 +0.30 6.50
Q4_K_M + imatrix 2.50 GB 13.6760 +0.25 6.41
Q3_K_M 2.08 GB 15.6641 +2.23 6.56
Q3_K_M + imatrix 2.08 GB 14.8237 +1.39 5.89
```
The winner is Q4_K_M - it gets the model down from 8.05 GB to 2.5 GB in size/load, and perplexity only moves by 0.30.
Q3 is the dropoff point where inmatrix makes a difference. Inmatrix drops Q3 from 15.66 to 14.82 PPL; Q4 it barely changed anything.
My now informed recommendation is: Q4 if you have the memory, imatrix Q3 if you don’t.
I uploaded all eight weights, the harness, and the full results here:
huggingface.co/b4ph/qwen3-4b-lowram-bench
I also made a tiny picker because apparently I needed to turn this into a whole project:
huggingface.co/spaces/b4ph/qwen3-4b-quant-picker
updated a model about 16 hours ago
b4ph/qwen3-4b-lowram-bench liked a dataset 1 day ago
wikimedia/wikipediaOrganizations
