SeaWolf-AI (VIDRAFT

replied to their post about 4 hours ago

We want that board too. We cannot open it on the data we have.

ChEMBL gives us 95 logBB records, 983 Kp, 13 B/P. Collapse those to unique
compounds and cut by year and the test set lands well under our floor of 150.
Worse, almost no compound appears in two separate publications, so we cannot
measure an experimental noise floor for it.

Without a noise floor we cannot say whether a gap between two entries is skill
or luck, and a leaderboard that cannot say that is ranking luck. So it stays shut.

If you know a source with a clear licence and cross-publication repeats, tell us
and we will open it.

replied to their post about 7 hours ago

Two corrections.

The constant-predictor result is not a broken split. On the same data with a
random split the trained model wins by a wide margin (MAE 0.457 vs 0.671).
Under a time split it does not. Both numbers are in the linked write-up. That
gap is the finding, and it is why we publish untrained baselines on every board.

The materials challenge states which axes it scores, in public, before entry.
Naming what a season does not measure is disclosure, not a defect.

reacted to their post with ❤️ about 15 hours ago

Post

1173

We opened a benchmark for drug property prediction tools. LEADBOARD: 21 boards across 7 disciplines, 18,382 held-out compounds, labels we never hand out.

Two numbers we hit while building it are the reason it exists.

First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.

Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.

So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.

Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.

Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard

5 replies

·

posted an update about 15 hours ago

Post

1173

We opened a benchmark for drug property prediction tools. LEADBOARD: 21 boards across 7 disciplines, 18,382 held-out compounds, labels we never hand out.

Two numbers we hit while building it are the reason it exists.

First. Split the hERG cardiotoxicity data at random and you get AUROC 0.818. Split it by first-report year instead and you get 0.606. Same molecules, same fingerprints, same learner, same hyperparameters. The only thing that changed was where the line went, and the score moved 0.211. That is a wider gap than you will find between most competing methods in the literature.

Second. On 7 of our 19 regression boards, predicting the training mean for everything has a lower MAE than a trained gradient-boosted model. hERG is one of them, 0.599 against 0.589. The trained model loses.

So every board publishes its homework before anyone submits. Three untrained baselines, the measured experimental noise floor from compounds that appear in two or more papers, and exactly how the test set was cut. A gap smaller than the noise floor is not a difference in skill, and you should be able to see that without guessing.

Entering is simple. Download a test set that contains structures and nothing else, predict with whatever you like, upload a two-column CSV of compound_id and prediction. Trained model, physics engine, LLM, rule of thumb. We do not care what is inside. We measure the output.

Post: https://huggingface.co/blog/FINAL-Bench/leadboard-drug
Leaderboard: FINAL-Bench/leadboard

5 replies

·

reacted to their post with 😎🤗👀➕ 1 day ago

Post

3295

🔋 Open Materials Challenge, Season 1 — Solid-State Battery Electrolytes

A solid-state battery replaces the liquid electrolyte of a lithium-ion cell with a solid. It does not catch fire, it lasts longer, and it can hold more. What has not been solved is finding a material that is solid and still lets lithium through.

Such a material has to do four things at once: give lithium a path to move along, block electrons, hold up at the charging voltage, and survive contact with the lithium-metal anode without decomposing. Plenty of materials manage three. Very few manage all four.

This challenge looks for candidates, together. You submit one composition — for example Li3YCl6. We score it computationally and place it on the board. There is no prize.

Scoring (100 points)

Oxidation stability 40 does it resist decomposing as the voltage rises
Lithium-metal stability 35 does it survive contact with the anode
Use novelty 25 higher if it has not been reported as an electrolyte
Entry condition a percolating path for lithium must exist

Ionic conductivity is not a scored axis this season. Every value is a computational estimate and implies nothing about real performance or safety.

The board also carries seven electrolytes in actual use — LGPS, argyrodite, LLZO, LATP and others. They are scored but hold no rank. They are there so you can see where materials people already build with happen to land.

Compositions are private by default. Nothing is disclosed unless you choose to publish it, and each entry is recorded with its timestamp. If a third party asks to discuss a particular entry, we pass the request along — never the submitter's identity, unless they agree to it.

Season 1 runs 2026-08-21 to 11-30. A participation guide and a set of prompts are included.

👉 FINAL-Bench/open-materials-challenge

Our sister challenge🧪https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge

reacted to their post with ➕🤝😎🚀👍❤️ 1 day ago

Post

3925

3,631 candidate molecules arrived in five days, from 83 accounts — roughly 700 a day. Far more than we expected. Thank you.

Yesterday we opened the third season and 224 arrived within a day: Chagas disease.

Why this disease

Around 6 million people live with it, mostly in Latin America (WHO). Many carry it for decades without knowing, while the heart is slowly damaged. There are two drugs and both date from the 1960s, hard enough to tolerate that many patients cannot finish the two-month course.

Sixty years without a new drug is not only a scientific problem. Most patients live where development costs cannot be recovered, which is why WHO calls this a neglected tropical disease.

But the cost of proposing a candidate and filtering it has changed. So it seemed worth asking whether work nobody funds could be done by many people sharing it out.

The problem this season

The target is CYP51, the enzyme T. cruzi uses to build its membrane sterols. Block it and the parasite cannot survive. The difficulty is that we carry the same enzyme.

Scoring: binding 30 · selectivity 30 · ADMET 15 · whole-cell 10 · novelty 10 · synthesis 5

Selectivity carries 30 points because nobody has solved it. Among the approved azoles on the board as reference compounds, some score 0 on selectivity — not a scorer fault, but the measurement.

Taking part

Design with any model, submit a SMILES, scored within minutes. Five ready-to-paste prompts per season, and the full rubric is published. Your molecule stays yours; private submission is the default.

Prizes — 4,000 USD across three seasons

Malaria 30 Sep · 1,000 | Tuberculosis 31 Oct · 2,000 | Chagas 30 Nov · 1,000

We know this does not cover the time you spend. It is a way of saying the work had worth.

FINAL-Bench/open-discovery-challenge

reacted to their post with 🧠👍🚀❤️🔥 1 day ago

Post

3295

🔋 Open Materials Challenge, Season 1 — Solid-State Battery Electrolytes

A solid-state battery replaces the liquid electrolyte of a lithium-ion cell with a solid. It does not catch fire, it lasts longer, and it can hold more. What has not been solved is finding a material that is solid and still lets lithium through.

Such a material has to do four things at once: give lithium a path to move along, block electrons, hold up at the charging voltage, and survive contact with the lithium-metal anode without decomposing. Plenty of materials manage three. Very few manage all four.

This challenge looks for candidates, together. You submit one composition — for example Li3YCl6. We score it computationally and place it on the board. There is no prize.

Scoring (100 points)

Oxidation stability 40 does it resist decomposing as the voltage rises
Lithium-metal stability 35 does it survive contact with the anode
Use novelty 25 higher if it has not been reported as an electrolyte
Entry condition a percolating path for lithium must exist

Ionic conductivity is not a scored axis this season. Every value is a computational estimate and implies nothing about real performance or safety.

The board also carries seven electrolytes in actual use — LGPS, argyrodite, LLZO, LATP and others. They are scored but hold no rank. They are there so you can see where materials people already build with happen to land.

Compositions are private by default. Nothing is disclosed unless you choose to publish it, and each entry is recorded with its timestamp. If a third party asks to discuss a particular entry, we pass the request along — never the submitter's identity, unless they agree to it.

Season 1 runs 2026-08-21 to 11-30. A participation guide and a set of prompts are included.

👉 FINAL-Bench/open-materials-challenge

Our sister challenge🧪https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge

posted an update 1 day ago

Post

3295

🔋 Open Materials Challenge, Season 1 — Solid-State Battery Electrolytes

A solid-state battery replaces the liquid electrolyte of a lithium-ion cell with a solid. It does not catch fire, it lasts longer, and it can hold more. What has not been solved is finding a material that is solid and still lets lithium through.

Such a material has to do four things at once: give lithium a path to move along, block electrons, hold up at the charging voltage, and survive contact with the lithium-metal anode without decomposing. Plenty of materials manage three. Very few manage all four.

This challenge looks for candidates, together. You submit one composition — for example Li3YCl6. We score it computationally and place it on the board. There is no prize.

Scoring (100 points)

Oxidation stability 40 does it resist decomposing as the voltage rises
Lithium-metal stability 35 does it survive contact with the anode
Use novelty 25 higher if it has not been reported as an electrolyte
Entry condition a percolating path for lithium must exist

Ionic conductivity is not a scored axis this season. Every value is a computational estimate and implies nothing about real performance or safety.

The board also carries seven electrolytes in actual use — LGPS, argyrodite, LLZO, LATP and others. They are scored but hold no rank. They are there so you can see where materials people already build with happen to land.

Compositions are private by default. Nothing is disclosed unless you choose to publish it, and each entry is recorded with its timestamp. If a third party asks to discuss a particular entry, we pass the request along — never the submitter's identity, unless they agree to it.

Season 1 runs 2026-08-21 to 11-30. A participation guide and a set of prompts are included.

👉 FINAL-Bench/open-materials-challenge

Our sister challenge🧪https://huggingface.co/spaces/FINAL-Bench/open-discovery-challenge

VIDRAFT_LAB

AI & ML interests

Recent Activity

Organizations

VIDRAFT_LAB

AI & ML interests

Recent Activity

Organizations

SeaWolf-AI's activity