Nanbeige/Nanbeige4.1-3B
Text Generation
ā¢
4B
ā¢
Updated
ā¢
21k
ā¢
ā¢
508
Manuel Barca-Grama
jackangel
AI & ML interests
None yet
Recent Activity
liked
a model
2 days ago
Nanbeige/Nanbeige4.1-3B
liked
a model
4 days ago
inclusionAI/ZwZ-8B
liked
a model
4 days ago
mradermacher/Rio-3.0-Open-Mini-i1-GGUF
Organizations
None yet
liked
a
model
2 days ago
liked
2
models
4 days ago
reacted to
marksverdhei's
post with š
7 days ago
Post
4499
Poll: Will 2026 be the year of subquadratic attention?
The transformer architecture is cursed by its computational complexity.
It is why you run out of tokens and have to compact. But some would argue that this is a feature not a bug and that this is also why these models are so good. We've been doing a lot of research on trying to make equally good models that are computationally cheaper, But so far, none of the approaches have stood the test of time. Or so it seems.
Please vote, don't be shy. Remember that the Dunning-Kruger effect is very real, so the person who knows less about transformers than you is going to vote. We want everyone's opinion, no matter confidence.
š if you think at least one frontier model* will have no O(n^2) attention by the end of 2026
š„ If you disagree
* Frontier models - models that match / outperform the flagship claude, gemini or chatgpt at the time on multiple popular benchmarks
The transformer architecture is cursed by its computational complexity.
It is why you run out of tokens and have to compact. But some would argue that this is a feature not a bug and that this is also why these models are so good. We've been doing a lot of research on trying to make equally good models that are computationally cheaper, But so far, none of the approaches have stood the test of time. Or so it seems.
Please vote, don't be shy. Remember that the Dunning-Kruger effect is very real, so the person who knows less about transformers than you is going to vote. We want everyone's opinion, no matter confidence.
š if you think at least one frontier model* will have no O(n^2) attention by the end of 2026
š„ If you disagree
* Frontier models - models that match / outperform the flagship claude, gemini or chatgpt at the time on multiple popular benchmarks
- 4 replies
liked
a
model
7 days ago
liked
2
models
10 days ago
liked
a
model
15 days ago
liked
a
model
19 days ago
liked
a
model
21 days ago
liked
a
model
29 days ago
liked
3
models
about 1 month ago
liked
a
dataset
about 1 month ago
liked
4
models
about 1 month ago
nvidia/nemotron-speech-streaming-en-0.6b
Automatic Speech Recognition
ā¢
Updated
ā¢
9.74k
ā¢
476
nvidia/Nemotron-Cascade-14B-Thinking
Text Generation
ā¢
Updated
ā¢
8.03k
ā¢
74
miromind-ai/MiroThinker-v1.5-30B
Text Generation
ā¢
Updated
ā¢
94.7k
ā¢
239
tiiuae/Falcon-H1R-7B-GGUF
Text Generation
ā¢
8B
ā¢
Updated
ā¢
6.29k
ā¢
69
liked
a
model
about 2 months ago
