Checkpoints for Fast-weight Product Key Memory (https://arxiv.org/abs/2601.00671)
AI & ML interests
We are a Tokyo-based R&D company on a quest to create a new kind of foundational AI model based on nature-inspired intelligence.
Recent Activity
Papers
View all PapersKAME
Extending the Context of Pretrained LLMs by Dropping Their Positional Embedding (https://www.arxiv.org/abs/2512.12167)
Students distilled from a 7B Reinforcement-Learned Teacher (RLT) from the paper "Reinforcement Learning Teachers of Test Time Scaling."
-
SakanaAI/Llama-3-8B-Instruct-CycleQD-CS
Text Generation • 8B • Updated • 88 • • 1 -
SakanaAI/Llama-3-8B-Instruct-Coding-Expert
Text Generation • 8B • Updated • 104 • • 13 -
SakanaAI/Llama-3-8B-Instruct-DB-Expert
Text Generation • 8B • Updated • 88 • -
SakanaAI/Llama-3-8B-Instruct-OS-Expert
Text Generation • 8B • Updated • 94 • • 1
Sparser, Faster, Lighter Transformer Language Models
because thought takes time and reasoning is a process.
Compact Japanese models trained with "TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models"
-
SakanaAI/TinySwallow-1.5B
Text Generation • 2B • Updated • 294 • • 34 -
SakanaAI/TinySwallow-1.5B-Instruct
Text Generation • 2B • Updated • 1.24k • • 59 -
SakanaAI/TinySwallow-1.5B-Instruct-q4f32_1-MLC
Text Generation • Updated • 3 -
SakanaAI/TinySwallow-1.5B-Instruct-GGUF
Text Generation • 2B • Updated • 443 • 29
Checkpoints for Fast-weight Product Key Memory (https://arxiv.org/abs/2601.00671)
KAME
Sparser, Faster, Lighter Transformer Language Models
Extending the Context of Pretrained LLMs by Dropping Their Positional Embedding (https://www.arxiv.org/abs/2512.12167)
because thought takes time and reasoning is a process.
Students distilled from a 7B Reinforcement-Learned Teacher (RLT) from the paper "Reinforcement Learning Teachers of Test Time Scaling."
Compact Japanese models trained with "TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models"
-
SakanaAI/TinySwallow-1.5B
Text Generation • 2B • Updated • 294 • • 34 -
SakanaAI/TinySwallow-1.5B-Instruct
Text Generation • 2B • Updated • 1.24k • • 59 -
SakanaAI/TinySwallow-1.5B-Instruct-q4f32_1-MLC
Text Generation • Updated • 3 -
SakanaAI/TinySwallow-1.5B-Instruct-GGUF
Text Generation • 2B • Updated • 443 • 29
-
SakanaAI/Llama-3-8B-Instruct-CycleQD-CS
Text Generation • 8B • Updated • 88 • • 1 -
SakanaAI/Llama-3-8B-Instruct-Coding-Expert
Text Generation • 8B • Updated • 104 • • 13 -
SakanaAI/Llama-3-8B-Instruct-DB-Expert
Text Generation • 8B • Updated • 88 • -
SakanaAI/Llama-3-8B-Instruct-OS-Expert
Text Generation • 8B • Updated • 94 • • 1