FINAL-Bench/VKORE-325M-NPU-Base
Text Generation โข 0.3B โข Updated โข 49 โข 14
Proto_AGI
mayafree
AI & ML interests
None yet
Recent Activity
liked a model about 8 hours ago
FINAL-Bench/VKORE-325M-NPU-Base upvoted an article about 8 hours ago
Pretraining a 325M Language Model from Zero on an Inference-Only NPU reacted to SeaWolf-AI's post with ๐ about 8 hours ago
We pretrained a 325M language model from zero on an inference-only NPU
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
๐ Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
๐ค Models: https://huggingface.co/FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKOREOrganizations
liked a
model about 8 hours ago
upvoted an article about 8 hours ago
Article
Pretraining a 325M Language Model from Zero on an Inference-Only NPU
FINAL-Bench
โข โข 7 reacted to SeaWolf-AI's post with ๐๐๐ฅ about 8 hours ago
Post
1,276
We pretrained a 325M language model from zero on an inference-only NPU
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
๐ Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
๐ค Models: FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKORE
The accepted view is that inference-only NPUs cannot train, let alone pretrain a language model from scratch. We put that view to a direct test on the Rebellions ATOM-Max, a commercially available Korean NPU released in 2025 and designed for inference. We built our own training loop that runs backpropagation on the chip itself. Starting from random initialization, we pretrained VKORE-325M-NPU-Base, a 325.6M parameter model, on 5 billion byte tokens of FineWeb-Edu. The run took 76,292 steps over about four days and finished without a single divergence. No pretrained weights were used.
To make the comparison fair, we trained the same model on a B200 GPU with identical initialization, data and learning-rate schedule. On 1,024 windows of text that neither model ever saw, the NPU reached a loss of 0.7993 and the GPU 0.7994. The window-by-window difference is -0.0001, well inside the noise. The two models are indistinguishable.
This is seven times larger than our first NPU model, VKORE-46M, and the first time this hardware has carried a real pretraining run to GPU quality. Please note that the hardware alone does not make this possible: the NPU and its standard software are built for inference, and training on it depends on our own training stack and stability technique, which are developed in house and under patent application. Buying the chip does not by itself enable training. Both sets of weights and the evaluation script are public, so anyone can verify the result in minutes.
๐ Technical report: https://huggingface.co/blog/FINAL-Bench/pretraining-a-325m-language-model-from-zero-on-an
๐ค Models: FINAL-Bench/VKORE-325M-NPU-Base
#NPU #Rebellions #Pretraining #VKORE
- 2 replies
reacted to SeaWolf-AI's post with ๐๐ 12 days ago
Post
2,776
๐ We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
๐ค Model: FINAL-Bench/VKORE-46M-NPU ๐ Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE ยท SwiGLU ยท RMSNorm ยท GQA), char-level, TinyStories.
๐ Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. ยท Loss trajectories correlate at 0.99 ยท Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
๐งฉ One accelerator now covers pretraining ยท LoRA ยท inference ยท quantization. We call this layer VKORE: "CUDA for NPUs."
โ๏ธ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. ๐
๐ค Model: FINAL-Bench/VKORE-46M-NPU ๐ Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE ยท SwiGLU ยท RMSNorm ยท GQA), char-level, TinyStories.
๐ Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. ยท Loss trajectories correlate at 0.99 ยท Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
๐งฉ One accelerator now covers pretraining ยท LoRA ยท inference ยท quantization. We call this layer VKORE: "CUDA for NPUs."
โ๏ธ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. ๐
- 10 replies
upvoted an article 12 days ago
Article
Backpropagation on an Inference-Only NPU: Pretraining VKORE-46M-NPU From Scratch
FINAL-Bench
โข โข 14 liked a
Space 12 days ago
liked a
model 12 days ago
reacted to SeaWolf-AI's post with ๐ฅ 12 days ago
Post
2,776
๐ We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
๐ค Model: FINAL-Bench/VKORE-46M-NPU ๐ Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE ยท SwiGLU ยท RMSNorm ยท GQA), char-level, TinyStories.
๐ Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. ยท Loss trajectories correlate at 0.99 ยท Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
๐งฉ One accelerator now covers pretraining ยท LoRA ยท inference ยท quantization. We call this layer VKORE: "CUDA for NPUs."
โ๏ธ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. ๐
๐ค Model: FINAL-Bench/VKORE-46M-NPU ๐ Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE ยท SwiGLU ยท RMSNorm ยท GQA), char-level, TinyStories.
๐ Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. ยท Loss trajectories correlate at 0.99 ยท Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
๐งฉ One accelerator now covers pretraining ยท LoRA ยท inference ยท quantization. We call this layer VKORE: "CUDA for NPUs."
โ๏ธ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. ๐
- 10 replies
liked 2
Spaces 15 days ago
liked a
dataset 15 days ago
upvoted an article 15 days ago
Article
The Open Quantum Challenge: Quantum Simulation and QEC Decoding on Classical GPUs
FINAL-Bench
โข โข 11 liked a
Space 15 days ago
reacted to SeaWolf-AI's post with ๐ค 15 days ago
Post
4,649
โ๏ธ The Open Quantum Challenge is live. Both seasons open today.
Quantum computers are expensive, queued, and reachable by only a few. So we removed the barrier. With a single GPU and a public harness, anyone can contribute to core quantum-computing problems with no quantum hardware at all. That is the point of this challenge: to move quantum research from a handful of labs to everyone.
โข Season 1, Quantum simulation: reproduce a target quantum system on a GPU.
โข Season 2, QEC decoder: correct errors on coded quantum states.
Answers are held privately and every submission is auto-scored against a frozen ground truth, so the ranking is reproducible and hardware-independent.
๐ Prize: 2,000 USD total (1,000 USD per season, ๐ฅ600 / ๐ฅ300 / ๐ฅ100)
Getting started takes one step: copy the participation guide from the Space and paste it into Codex or Claude Code, then submit against the harness.
๐ Read the article: https://huggingface.co/blog/FINAL-Bench/the-open-quantum-challenge-quantum-simulation-and
๐ Challenge: FINAL-Bench/open-quantum-challenge
Quantum computers are expensive, queued, and reachable by only a few. So we removed the barrier. With a single GPU and a public harness, anyone can contribute to core quantum-computing problems with no quantum hardware at all. That is the point of this challenge: to move quantum research from a handful of labs to everyone.
โข Season 1, Quantum simulation: reproduce a target quantum system on a GPU.
โข Season 2, QEC decoder: correct errors on coded quantum states.
Answers are held privately and every submission is auto-scored against a frozen ground truth, so the ranking is reproducible and hardware-independent.
๐ Prize: 2,000 USD total (1,000 USD per season, ๐ฅ600 / ๐ฅ300 / ๐ฅ100)
Getting started takes one step: copy the participation guide from the Space and paste it into Codex or Claude Code, then submit against the harness.
๐ Read the article: https://huggingface.co/blog/FINAL-Bench/the-open-quantum-challenge-quantum-simulation-and
๐ Challenge: FINAL-Bench/open-quantum-challenge
reacted to SeaWolf-AI's post with ๐๐๐ 16 days ago
Post
4,123
๐ง We just released Darwin-27B-ZTC, a judgment engine that reaches a verdict without generating anything.
Most LLMs answer by generating, decoding one token at a time. Darwin-27B-ZTC takes a different route.
โ๏ธ How it works
๐น It makes its call in a single forward pass.
๐น Zero generated tokens, and no decoding loop.
๐น That keeps latency and cost far below what a generative model needs.
๐ฏ What it judges
๐น It handles several question types: free-form correctness (noul), multiple choice (choice), and scoring (score).
๐น For each one it hands back a calibrated confidence, not just an answer.
๐ How well calibrated (measured)
๐น KL 0.204, Brier 0.097, so the confidence it reports lines up with what actually happens.
๐น 0.743 accuracy (zero-shot, general split), across 2,000 judgments with zero errors.
๐น By type: noul 0.847, choice 0.723, score 0.675.
๐น None of the benchmark's train split went into it. It is pure zero-shot.
๐ Where it fits
๐น Grading at scale, model routing, safety gating, anywhere you want a fast decision without paying for generation.
๐ It currently sits at #1 on the official typed-decisions leaderboard on Hugging Face (0.743 accuracy, zero-shot).
๐ Links
Model: FINAL-Bench/Darwin-27B-ZTC
Leaderboard: LocalLLaMA/typed-decisions
Curious to hear what you make of the single-pass, no-generation approach. ๐
Most LLMs answer by generating, decoding one token at a time. Darwin-27B-ZTC takes a different route.
โ๏ธ How it works
๐น It makes its call in a single forward pass.
๐น Zero generated tokens, and no decoding loop.
๐น That keeps latency and cost far below what a generative model needs.
๐ฏ What it judges
๐น It handles several question types: free-form correctness (noul), multiple choice (choice), and scoring (score).
๐น For each one it hands back a calibrated confidence, not just an answer.
๐ How well calibrated (measured)
๐น KL 0.204, Brier 0.097, so the confidence it reports lines up with what actually happens.
๐น 0.743 accuracy (zero-shot, general split), across 2,000 judgments with zero errors.
๐น By type: noul 0.847, choice 0.723, score 0.675.
๐น None of the benchmark's train split went into it. It is pure zero-shot.
๐ Where it fits
๐น Grading at scale, model routing, safety gating, anywhere you want a fast decision without paying for generation.
๐ It currently sits at #1 on the official typed-decisions leaderboard on Hugging Face (0.743 accuracy, zero-shot).
๐ Links
Model: FINAL-Bench/Darwin-27B-ZTC
Leaderboard: LocalLLaMA/typed-decisions
Curious to hear what you make of the single-pass, no-generation approach. ๐
- 2 replies