common-pile/project_gutenberg_filtered
Viewer • Updated • 57.1k • 2.73k • 7
alkinun
AtAndDev
AI & ML interests
decentralize
Recent Activity
liked a dataset 2 days ago
common-pile/project_gutenberg_filtered liked a dataset 3 days ago
IFM/Pretrain-Behaviors reacted to juiceb0xc0de's post with 🔥 3 days ago
Dude, Where's My Update? I'll tell you where! ~97.6% of my BF16 parameter coordinates didn't move at all, and the ones that did overshot by ~1.33x.
It's nice to do research that doesn't end in disproving yourself once again and moving on to the next subject once in awhile.
Back to the topic, if you've ever wondered why most of your weights are basically ghosting you nearly every step when you store your weights at bf16, Dude, I Measured It.
https://huggingface.co/blog/juiceb0xc0de/intended-and-realized-updates-in-bf16-fine-tuning#dude-wheres-my-updateOrganizations
liked a
dataset 2 days ago
liked 2
datasets 3 days ago
liked a
Space 3 days ago
liked a
dataset 4 days ago
liked a
model 5 days ago
liked a
dataset 7 days ago
liked 2
datasets 8 days ago
liked a
model 8 days ago
liked 3
datasets 9 days ago
liked a
model 9 days ago
liked 4
models 10 days ago
liked a
dataset 11 days ago
liked a
Space 11 days ago