Hugging Face's logo
appvoid's picture
🤏 quecto mode

appvoid

appvoid
58 4 182
Andhika1996's profile picture CodeSoft's profile picture Algiefonso's profile picture
·
https://ko-fi.com/appvoid

AI & ML interests

singularity through byte-level tokens and small language models - creating applications and ideas out of the void

Recent Activity

posted an update about 1 hour ago
We trained a 10.9M byte-level recurrent Transformer on L3 and L6. (Loop 3 and Loop 6) Yet L4/L5 improved too, L8 held up, and the L3→L6 gain grew during training. Same weights. More compute. Better predictions. This is a new architecture for effective compute after several steps beyond original training! We mixed and matched components like time and mhc into an ouro-like byte-level language model and the result is BET, a byte-level step-elastic transformer that can run computation steps without significant degradation. One of the coolest parts of this training was discovering how Gradient Descent decided to use the first layer as what we would consider a scratchpad! Totally destroyed for the decoder but somehow makes total sense for the next layer! I believe looped-transformers are the future of edge computing and this is a first step towards it. Blogpost: https://medium.com/@appvoidofficial/byte-level-elasticity-182fe2ed1d2f https://huggingface.co/appvoid/bet-10m
updated a collection about 2 hours ago
releases
published a model about 2 hours ago
appvoid/bet-10m
View all activity

Organizations

ZeroGPU Explorers's profile picture Social Post Explorers's profile picture BananaMind Model Previewers's profile picture Community Early Access Models from appvoid's profile picture The Dot Company's profile picture Muse Labs's profile picture