Post
10774
THIS IS CRAZY! THE MODEL ON THE IMAGE(Supra-50M-Reasoning) answered correctly and its QUANTIZED IN 2BIT! THE RESPONSE IS CORRECT, IN A 15MB SIZE FILE!
Join the conversation
Join the community of Machine Learners and AI enthusiasts.
Sign Up
Yes it is , Tell me is the model consistent with it ? Like out of how many prompt did model get right answer ?
I mean, like, we are talking about a 50M model here. That thing, already in full precision, can barely do basic math. On 2 bits, even a single question being right is impressive.
Great
interesting
It got my His information wrong—and this was INT8. Unworthy.
But seriously lmao what is this??
It turns out that you can cherry pick results, post an image, and act like it generalizes
I've wondered, if a sufficiently reasoned model can get away with missing a lot of data because it can reason it out, and thus the smaller size may be worth it to be faster/leaner (even if it needs more tokens to reason the solution).
But still struggling for me, HEAVILY! Even the Minecraft question DOESN'T ANSWER IT RIGHT! AND YES, I TRIED THE INSTRUCT VERSION AND FAILS HEAVILY!
Believe it or not, questions revolving games can be the hardest for AIs to solve, as you require the knowledge of actually playing to game to truly know the game. By that I mean, just think of these two questions: Do you learn from experience or from what you are straightforwardly told? Do you learn in math class by just listening or participating?
A good conclusion from this is that AI learns not visually but from already observed patterns and relationships at a reasonable scale. That's why AI can be really good at math; terrible at what comes naturally to us.
That's just optimizers, that's just human learning, both sound similar, both far from each other.
Guys its a 50m model 😭 Its not gonna be factually correct or reliable, the fact that it works at all is very impressive