POCKET vs Bonsai · CPU
⚔
26
BONSAI vs POCKET — 35B MoE out-runs the top 1-bit 27B on CPU
YoungMin
Rickie77
AI & ML interests
None yet
Recent Activity
liked a Space about 11 hours ago
FINAL-Bench/POCKET-35B-CPU reacted to SeaWolf-AI's post with ❤️ about 12 hours ago
📱 POCKET — a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): https://huggingface.co/FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU
📚 Collection: https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
reacted to SeaWolf-AI's post with 🤗 about 12 hours ago
📱 POCKET — a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): https://huggingface.co/FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): https://huggingface.co/FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): https://huggingface.co/FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): https://huggingface.co/FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): https://huggingface.co/spaces/FINAL-Bench/POCKET-35B-CPU
📚 Collection: https://huggingface.co/collections/FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
Organizations
None yet
liked a
Space about 11 hours ago
reacted to SeaWolf-AI's post with ❤️🤗👍 about 12 hours ago
Post
1224
📱 POCKET — a 35-billion-parameter model that runs on your iPhone, and on your PC with no GPU
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
📚 Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
We're releasing POCKET, VIDRAFT's flagship Darwin-36B-Opus compressed for on-device use. No fork, no CUDA, no cloud — it runs on stock llama.cpp. It's a sparse Mixture-of-Experts model (256 experts, only 8 active per token), so the file can be large while the work per token stays small. That's what lets a 35B model run on a phone, and generate fast on a CPU with no graphics card.
Measured (POCKET-35B IQ1_M vs Bonsai-27B Q1_0):
• CPU generate (Xeon, 16 threads): 27.0 vs 10.1 tok/s → 2.69× faster
• GPU generate (H100): 197 vs 89 tok/s → 2.22× faster
• GPU prompt processing (H100): 753 vs 1816 → 0.41× (Bonsai wins this one — MoE prefill wakes every expert, so sparsity stops helping there. We say so.)
• Quality (HellaSwag, 400 q): 61.0% vs 60.0% → a tie (confidence intervals overlap)
On a real consumer laptop — MacBook M3 Pro (18 GB) — POCKET wins every axis, prompt processing included:
• Metal generate: 25.4 vs 12.8 → 1.99×
• CPU generate: 13.8 vs 4.4 → 3.13×
• Metal prompt: 240.7 vs 73.4 → 3.28×
One more quiet fact: the same-size, quality-oriented rival Ternary-Bonsai-27B (7.2 GB) fails to load in upstream llama.cpp at all — it needs the PrismML fork. POCKET runs on the tools you already have: LM Studio, Ollama, PocketPal, MLX.
📖 Full story (tech, measurements, recipes): https://huggingface.co/blog/FINAL-Bench/pocket
Models:
📦 POCKET-35B-GGUF (PC / server, no GPU): FINAL-Bench/POCKET-35B-GGUF
🇰🇷 POCKET-KR-GGUF (Android): FINAL-Bench/POCKET-KR-GGUF
🍎 POCKET-KR-MLX (iPhone / Mac): FINAL-Bench/POCKET-KR-MLX
🌍 POCKET-EN-GGUF (English phone / PC): FINAL-Bench/POCKET-EN-GGUF
🖥️ Live demo (answering on a CPU, no GPU): FINAL-Bench/POCKET-35B-CPU
📚 Collection: FINAL-Bench/pocket-models-6a618ee5d23eafb7e185a5c6
reacted to SeaWolf-AI's post with 🚀🔥❤️ 4 days ago
Post
5104
A small gift for anyone building or studying foundation models.
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing — so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
🔁 Rebuild it from scratch, or fork the recipe for your own model
🔬 Study a real heterogeneous-attention MoE — 49 layers place 5 attention mechanisms on a 7×7 Latin square, arranged as a clean, attributable ablation
📈 Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one — the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. 🤗
📖 Full write-up →
[blog] · https://huggingface.co/blog/FINAL-Bench/opensource-llm
📦 5 Attention Base · FINAL-Bench/Aether-7B-5Attn
🎯 5 Attention Instruct · FINAL-Bench/Aether-7B-5Attn-it
🚀 5 Attention Live demo · FINAL-Bench/Aether-Sovereign-AI
📦 7 Attention Base · https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
📦 11 Attention Base · FINAL-Bench/Aether-6B-11Attn-base
🧬 Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing — so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
🔁 Rebuild it from scratch, or fork the recipe for your own model
🔬 Study a real heterogeneous-attention MoE — 49 layers place 5 attention mechanisms on a 7×7 Latin square, arranged as a clean, attributable ablation
📈 Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one — the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. 🤗
📖 Full write-up →
[blog] · https://huggingface.co/blog/FINAL-Bench/opensource-llm
📦 5 Attention Base · FINAL-Bench/Aether-7B-5Attn
🎯 5 Attention Instruct · FINAL-Bench/Aether-7B-5Attn-it
🚀 5 Attention Live demo · FINAL-Bench/Aether-Sovereign-AI
📦 7 Attention Base · https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
📦 11 Attention Base · FINAL-Bench/Aether-6B-11Attn-base
🧬 Collection · https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
- 5 replies
liked a
dataset 4 days ago
liked a
model 4 days ago
liked a
Space 4 days ago
upvoted a collection 4 days ago
upvoted an article 4 days ago
Article
Aether-7B-5Attn: A 100% Open-Source Sovereign Foundation Model — and a Controlled Experiment in Heterogeneous Attention
FINAL-Bench
• • 21 reacted to SeaWolf-AI's post with 👍❤️🔥 11 days ago
Post
5392
🔵 VKUE — No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound — sweep 34B params/token and an 8 GB card dies at 1–2 tok/s.
So we ran ONE 34.7B reasoning model — Ourbox-35B-JGOS, a sparse Mixture-of-Experts — as the identical weights across the whole hardware spectrum. All measured:
• B200: 18,057 tok/s (aggregate)
• 1× A10G: 126 tok/s
• 8 GB laptop (RTX 5060): 20 tok/s
• GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB — ~11× less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop — or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 → 3.7× from sparsity alone, ~2× the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
📝 Article: https://huggingface.co/blog/FINAL-Bench/vkue
🔵 GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
🔵 CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
📊 VKUE leaderboard: FINAL-Bench/VKUE
🤗 Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
⚡ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound — sweep 34B params/token and an 8 GB card dies at 1–2 tok/s.
So we ran ONE 34.7B reasoning model — Ourbox-35B-JGOS, a sparse Mixture-of-Experts — as the identical weights across the whole hardware spectrum. All measured:
• B200: 18,057 tok/s (aggregate)
• 1× A10G: 126 tok/s
• 8 GB laptop (RTX 5060): 20 tok/s
• GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB — ~11× less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop — or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 → 3.7× from sparsity alone, ~2× the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
📝 Article: https://huggingface.co/blog/FINAL-Bench/vkue
🔵 GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
🔵 CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
📊 VKUE leaderboard: FINAL-Bench/VKUE
🤗 Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
⚡ VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 7 replies
upvoted an article 11 days ago
Article
VKUE: No GPU? Runs Anyway — a 34.7B Reasoner on a Laptop and on Bare CPU
FINAL-Bench
• • 17 liked a
model 11 days ago
liked a
Space 11 days ago
upvoted an article 18 days ago
Article
Quantum Cryptanalysis on Real Hardware: Pushing Symmetric-Structure Key Recovery Beyond the Published Frontier
FINAL-Bench
• • 15 reacted to SeaWolf-AI's post with 😔 18 days ago
Post
5178
🔓 We ran genuine quantum key-recovery on 'real IBM quantum hardware' — and pushed the frontier well past the largest hardware demos we're aware of (which sat at N=4).
Using Simon's algorithm on
• Even–Mansour — N=5 → N=10
• 3-round Feistel (DES-family) — block 6 → 8
Each verified against an 'independent control key', using error mitigation only (no QEC).
🧭 Honest scope: this is not a quantum speedup (the effective difficulty tracks the classical birthday bound ~2^{n/2}), not a break of real AES/RSA, and not 16-round DES (ours is 3-round). The recovery method is reserved for a forthcoming paper; formal record status is pending peer review.
📄 Write-up: https://huggingface.co/blog/FINAL-Bench/quantum
🕹️ Try it live in your browser: https://vidraft-quantumos.hf.space/crypto
🏆 Leaderboard: FINAL-Bench/quantum-bench-leaderboard
#quantum #cryptography #quantumcomputing
Using Simon's algorithm on
ibm_kingston, we recovered the secret key of two symmetric-cipher structures:• Even–Mansour — N=5 → N=10
• 3-round Feistel (DES-family) — block 6 → 8
Each verified against an 'independent control key', using error mitigation only (no QEC).
🧭 Honest scope: this is not a quantum speedup (the effective difficulty tracks the classical birthday bound ~2^{n/2}), not a break of real AES/RSA, and not 16-round DES (ours is 3-round). The recovery method is reserved for a forthcoming paper; formal record status is pending peer review.
📄 Write-up: https://huggingface.co/blog/FINAL-Bench/quantum
🕹️ Try it live in your browser: https://vidraft-quantumos.hf.space/crypto
🏆 Leaderboard: FINAL-Bench/quantum-bench-leaderboard
#quantum #cryptography #quantumcomputing