Post
1333
Nobody knows what is doing, when you train a model, you are experimenting to advance the frontier, so keep failing 🫵
Join the conversation
Join the community of Machine Learners and AI enthusiasts.
Sign Up
Saying "Nobody knows what they are doing" is just a convenient excuse to justify terrible engineering. There is a fine line between scientific trial-and-error and proud, brute-force ignorance.Let's be clear about your "frontier":Blind Gambling: When independent labs don't understand the underlying mathematics or hardware physics, they just throw data at a wall and pray to the loss curve. That is digital alchemy, not science.The Loop: Instead of fixing structural bottlenecks or learning non-linear dynamics, people just brute-force configs. It’s the engineering equivalent of a cat grooming itself because it has nothing else to do.Zero Legacy: This unscientific approach is why the ecosystem is flooded with overfitted, hollow checkpoints that break down outside of their strict test sets.You aren't advancing the frontier; you are just polluting the platform because you refuse to open a textbook. Brute force has hit a physical wall. True innovation requires cognitive architecture and actual engineering, not just romanticizing failure. 🫵🤡
What is this guy even talking about
First, real indepedent labs aren't brute-forcing anything; we don't have the compute to do that. I mean, look at AxiomicLabs/GPT-2.5-135M, it gets within 2 points (via II score) of SmolLM2 with 60x less data. That is the exact opposite of brute force.
Second, in frontier deep learning, formal theory has always lagged results. Saying "no body knows what they are doing" simply ackolegdes (fuck i cant spell) that we are exploring uncharted territory with informed hypotheses rather than pretending a textbook already has all the answers.
100% true