Spaces:
Running
Running
feat(primo): Boards rail toggle, facet rename, ScientaLab data repos
Browse files- README.md +3 -3
- boards.py +2 -2
- evaluator.py +1 -1
- pages/about.md +2 -2
- pages/contribute.md +1 -1
- pages/submit.md +1 -1
- primo.css +14 -3
- quickstart.py +1 -1
- render.py +24 -3
README.md
CHANGED
|
@@ -37,7 +37,7 @@ that does not exist.
|
|
| 37 |
|
| 38 |
🌐 Website: http://primomics.org/ ·
|
| 39 |
📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
|
| 40 |
-
📦 Data: https://huggingface.co/datasets/
|
| 41 |
|
| 42 |
## What's in the data
|
| 43 |
|
|
@@ -133,8 +133,8 @@ pushed by `benchmark/public_benchmark/baselines.py --score --publish`.
|
|
| 133 |
- **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
|
| 134 |
Hugging Face* button; submitting requires a logged-in HF account.
|
| 135 |
- Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
|
| 136 |
-
`
|
| 137 |
-
`
|
| 138 |
`<task_id>/labels.csv`), and **read + write** on
|
| 139 |
`ScientaLab/primo-results` (the persisted leaderboard).
|
| 140 |
- Results persist as one normalized `task_results.csv` (`model_name, task_id,
|
|
|
|
| 37 |
|
| 38 |
🌐 Website: http://primomics.org/ ·
|
| 39 |
📄 Paper: https://openreview.net/forum?id=v2SA8gHwqo ·
|
| 40 |
+
📦 Data: https://huggingface.co/datasets/ScientaLab/primo
|
| 41 |
|
| 42 |
## What's in the data
|
| 43 |
|
|
|
|
| 133 |
- **`hf_oauth: true`** (set above) turns on the Submit tab's *Sign in with
|
| 134 |
Hugging Face* button; submitting requires a logged-in HF account.
|
| 135 |
- Set an **`HF_TOKEN`** Space secret (fine-grained) with: **read** on
|
| 136 |
+
`ScientaLab/primo` (the public `datasets.yaml` manifest) and
|
| 137 |
+
`ScientaLab/primo-labels` (the private `tasks.yaml` registry +
|
| 138 |
`<task_id>/labels.csv`), and **read + write** on
|
| 139 |
`ScientaLab/primo-results` (the persisted leaderboard).
|
| 140 |
- Results persist as one normalized `task_results.csv` (`model_name, task_id,
|
boards.py
CHANGED
|
@@ -37,8 +37,8 @@ MODALITY_LABEL = {
|
|
| 37 |
"single-cell RNA": "single-cell RNAseq",
|
| 38 |
}
|
| 39 |
|
| 40 |
-
MODALITY_GROUP = "
|
| 41 |
-
AREA_GROUP = "
|
| 42 |
CATEGORY_GROUP = "Task Category"
|
| 43 |
|
| 44 |
GROUP_NOTE = {
|
|
|
|
| 37 |
"single-cell RNA": "single-cell RNAseq",
|
| 38 |
}
|
| 39 |
|
| 40 |
+
MODALITY_GROUP = "Per modality"
|
| 41 |
+
AREA_GROUP = "Per therapeutic areas"
|
| 42 |
CATEGORY_GROUP = "Task Category"
|
| 43 |
|
| 44 |
GROUP_NOTE = {
|
evaluator.py
CHANGED
|
@@ -47,7 +47,7 @@ SPLIT = "split"
|
|
| 47 |
SPLIT_TRAIN = "train"
|
| 48 |
SPLIT_TEST = "test"
|
| 49 |
|
| 50 |
-
ORG = "
|
| 51 |
PUBLIC_REPO = f"{ORG}/primo"
|
| 52 |
LABELS_REPO = f"{ORG}/primo-labels"
|
| 53 |
# Keep the established leaderboard history available until it is migrated.
|
|
|
|
| 47 |
SPLIT_TRAIN = "train"
|
| 48 |
SPLIT_TEST = "test"
|
| 49 |
|
| 50 |
+
ORG = "ScientaLab"
|
| 51 |
PUBLIC_REPO = f"{ORG}/primo"
|
| 52 |
LABELS_REPO = f"{ORG}/primo-labels"
|
| 53 |
# Keep the established leaderboard history available until it is migrated.
|
pages/about.md
CHANGED
|
@@ -16,7 +16,7 @@ diseases**, and we ask the same three questions of each patient:
|
|
| 16 |
|
| 17 |
| Task family | The question | Metric |
|
| 18 |
|---|---|---|
|
| 19 |
-
| **Treatment outcome** | Will this patient respond to anti-TNF therapy? | AUROC |
|
| 20 |
| **Clinical scores** | How severe is this patient's disease? | Pearson r |
|
| 21 |
| **Endotype** | Which molecular subtype is this patient? | AUROC |
|
| 22 |
|
|
@@ -85,6 +85,6 @@ modality or therapeutic area, or work with us on the methodology? The
|
|
| 85 |
## Links
|
| 86 |
|
| 87 |
- 🌐 **Website**: [primomics.org](http://primomics.org/)
|
| 88 |
-
- 📦 **Data**: [
|
| 89 |
- 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 90 |
- 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
|
|
|
|
| 16 |
|
| 17 |
| Task family | The question | Metric |
|
| 18 |
|---|---|---|
|
| 19 |
+
| **Treatment outcome** | Will this patient respond to treatment (for example, anti-TNF therapy)? | AUROC |
|
| 20 |
| **Clinical scores** | How severe is this patient's disease? | Pearson r |
|
| 21 |
| **Endotype** | Which molecular subtype is this patient? | AUROC |
|
| 22 |
|
|
|
|
| 85 |
## Links
|
| 86 |
|
| 87 |
- 🌐 **Website**: [primomics.org](http://primomics.org/)
|
| 88 |
+
- 📦 **Data**: [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo)
|
| 89 |
- 📄 **Paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 90 |
- 💬 **Questions**: [karim.elkanbi@scientalab.com](mailto:karim.elkanbi@scientalab.com)
|
pages/contribute.md
CHANGED
|
@@ -53,4 +53,4 @@ methodology, co-signature on what we publish, and early access to the results.
|
|
| 53 |
- **Question the method.** Open a thread in the Space's **Community** tab.
|
| 54 |
- **Submit a model.** Baselines are on the leaderboard already; beat them.
|
| 55 |
- **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 56 |
-
- **Look at the data**: [
|
|
|
|
| 53 |
- **Question the method.** Open a thread in the Space's **Community** tab.
|
| 54 |
- **Submit a model.** Baselines are on the leaderboard already; beat them.
|
| 55 |
- **Read the paper**: [OpenReview](https://openreview.net/forum?id=v2SA8gHwqo)
|
| 56 |
+
- **Look at the data**: [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo)
|
pages/submit.md
CHANGED
|
@@ -15,7 +15,7 @@ python quickstart.py --out submission.parquet
|
|
| 15 |
|
| 16 |
Or do it by hand, in **three steps**:
|
| 17 |
|
| 18 |
-
1. **Get the data** → download the datasets from [
|
| 19 |
2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
|
| 20 |
3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
|
| 21 |
|
|
|
|
| 15 |
|
| 16 |
Or do it by hand, in **three steps**:
|
| 17 |
|
| 18 |
+
1. **Get the data** → download the datasets from [ScientaLab/primo](https://huggingface.co/datasets/ScientaLab/primo) (start with its `datasets.yaml`).
|
| 19 |
2. **Embed every dataset** → build **one** file: `dataset_id`, `sample_id`, then one column per embedding dim (`e0`, `e1`, …). CSV / TSV / Parquet, or NPZ.
|
| 20 |
3. **Sign in, fill the form, and hit Evaluate.** A fixed linear probe scores each hidden task (AUROC or Pearson), reported per task category in its native metric.
|
| 21 |
|
primo.css
CHANGED
|
@@ -59,7 +59,18 @@ footer { display: none !important; }
|
|
| 59 |
.pm-brand b { font: 700 1.125rem/1 var(--font-title); letter-spacing: -.04em; }
|
| 60 |
.pm-brand span { font: 500 1.25rem/0 var(--font-title); letter-spacing: -.04em; color: var(--sidebar-sub-fg); }
|
| 61 |
|
| 62 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
.pm-rail-label { display: flex; align-items: center; height: 2.5rem; padding: 8px;
|
| 64 |
font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--sidebar-sub-fg); }
|
| 65 |
|
|
@@ -86,11 +97,11 @@ footer { display: none !important; }
|
|
| 86 |
force the rail's inner text to take the link/brand colour instead. */
|
| 87 |
#pm-rail .pm-link .pm-label, #pm-rail .pm-link .pm-count, #pm-rail .pm-brand b { color: inherit !important; }
|
| 88 |
|
| 89 |
-
.pm-rail-foot { margin
|
| 90 |
#pm-rail .pm-foot-link {
|
| 91 |
display: flex !important; align-items: center; width: 100%; height: 2.5rem; padding: 8px; margin: 0 0 2px;
|
| 92 |
border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
|
| 93 |
-
font:
|
| 94 |
}
|
| 95 |
#pm-rail .pm-foot-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
|
| 96 |
#pm-rail .pm-foot-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; }
|
|
|
|
| 59 |
.pm-brand b { font: 700 1.125rem/1 var(--font-title); letter-spacing: -.04em; }
|
| 60 |
.pm-brand span { font: 500 1.25rem/0 var(--font-title); letter-spacing: -.04em; color: var(--sidebar-sub-fg); }
|
| 61 |
|
| 62 |
+
/* "Boards" collapses the facet groups so the foot links stay above the fold */
|
| 63 |
+
.pm-rail-section { margin: 8px 0 0; }
|
| 64 |
+
.pm-rail-summary { display: flex; align-items: center; justify-content: space-between; gap: 8px;
|
| 65 |
+
height: 2.5rem; padding: 8px; margin: 0; cursor: pointer; list-style: none;
|
| 66 |
+
border-radius: var(--radius-md); font: 600 13px/1 var(--font-sans); color: var(--sidebar-fg); }
|
| 67 |
+
.pm-rail-summary::-webkit-details-marker { display: none; }
|
| 68 |
+
.pm-rail-summary:hover { background: var(--sidebar-highlight); }
|
| 69 |
+
.pm-rail-summary.pm-active { background: var(--sidebar-accent); color: #fff; }
|
| 70 |
+
.pm-rail-summary .pm-chevron { flex: none; color: var(--sidebar-sub-fg); transition: transform .15s ease; }
|
| 71 |
+
.pm-rail-section[open] > .pm-rail-summary .pm-chevron { transform: rotate(90deg); }
|
| 72 |
+
|
| 73 |
+
.pm-rail-group { padding: 8px 16px 0; }
|
| 74 |
.pm-rail-label { display: flex; align-items: center; height: 2.5rem; padding: 8px;
|
| 75 |
font: 400 10px/1 var(--font-mono); letter-spacing: .08em; text-transform: uppercase; color: var(--sidebar-sub-fg); }
|
| 76 |
|
|
|
|
| 97 |
force the rail's inner text to take the link/brand colour instead. */
|
| 98 |
#pm-rail .pm-link .pm-label, #pm-rail .pm-link .pm-count, #pm-rail .pm-brand b { color: inherit !important; }
|
| 99 |
|
| 100 |
+
.pm-rail-foot { margin: 8px 0 0; padding: 12px 16px 16px; border-top: 1px solid var(--sidebar-border); }
|
| 101 |
#pm-rail .pm-foot-link {
|
| 102 |
display: flex !important; align-items: center; width: 100%; height: 2.5rem; padding: 8px; margin: 0 0 2px;
|
| 103 |
border: 0 !important; border-radius: var(--radius-md); background: transparent !important; box-shadow: none !important;
|
| 104 |
+
font: 600 14px/1 var(--font-sans); color: var(--sidebar-fg) !important; text-align: left;
|
| 105 |
}
|
| 106 |
#pm-rail .pm-foot-link:hover { background: var(--sidebar-highlight) !important; color: #fff !important; }
|
| 107 |
#pm-rail .pm-foot-link.pm-active { background: var(--sidebar-accent) !important; color: #fff !important; }
|
quickstart.py
CHANGED
|
@@ -22,7 +22,7 @@ import yaml
|
|
| 22 |
from huggingface_hub import snapshot_download
|
| 23 |
from sklearn.decomposition import PCA
|
| 24 |
|
| 25 |
-
PUBLIC_REPO = "
|
| 26 |
MANIFEST_FILENAME = "datasets.yaml"
|
| 27 |
|
| 28 |
DATASET_ID = "dataset_id"
|
|
|
|
| 22 |
from huggingface_hub import snapshot_download
|
| 23 |
from sklearn.decomposition import PCA
|
| 24 |
|
| 25 |
+
PUBLIC_REPO = "ScientaLab/primo"
|
| 26 |
MANIFEST_FILENAME = "datasets.yaml"
|
| 27 |
|
| 28 |
DATASET_ID = "dataset_id"
|
render.py
CHANGED
|
@@ -61,17 +61,37 @@ STRONG_COLUMNS = frozenset({"Task"})
|
|
| 61 |
METRIC_COLUMNS = frozenset({"Metric"})
|
| 62 |
RIGHT_COLUMNS = frozenset({"Rank", "Patients"})
|
| 63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
# --------------------------------------------------------------------- rail
|
| 66 |
def rail_html(
|
| 67 |
boards: list[Board], active_slug: str | None, active_tab: str | None
|
| 68 |
) -> str:
|
| 69 |
-
"""The left navigation: brand,
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
``active_slug`` highlights the board a visitor is on; ``active_tab`` highlights
|
| 72 |
-
a foot link
|
|
|
|
|
|
|
| 73 |
"""
|
| 74 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
for group in SECTIONS:
|
| 76 |
cards, opens = in_group(boards, group), open_in_group(boards, group)
|
| 77 |
if not cards and not opens:
|
|
@@ -93,6 +113,7 @@ def rail_html(
|
|
| 93 |
'<span class="pm-badge pm-badge--quiet">open</span></a>'
|
| 94 |
)
|
| 95 |
parts.append("</div>")
|
|
|
|
| 96 |
parts.append('<div class="pm-rail-foot">')
|
| 97 |
for tab, label in FOOT_LINKS:
|
| 98 |
cls = "pm-foot-link pm-active" if tab == active_tab else "pm-foot-link"
|
|
|
|
| 61 |
METRIC_COLUMNS = frozenset({"Metric"})
|
| 62 |
RIGHT_COLUMNS = frozenset({"Rank", "Patients"})
|
| 63 |
|
| 64 |
+
CHEVRON = (
|
| 65 |
+
'<svg class="pm-chevron" viewBox="0 0 16 16" width="12" height="12" '
|
| 66 |
+
'aria-hidden="true"><path d="M6 4l4 4-4 4" fill="none" stroke="currentColor" '
|
| 67 |
+
'stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round"/></svg>'
|
| 68 |
+
)
|
| 69 |
+
|
| 70 |
|
| 71 |
# --------------------------------------------------------------------- rail
|
| 72 |
def rail_html(
|
| 73 |
boards: list[Board], active_slug: str | None, active_tab: str | None
|
| 74 |
) -> str:
|
| 75 |
+
"""The left navigation: brand, a collapsible "Boards" menu, then the foot links.
|
| 76 |
+
|
| 77 |
+
The board groups live inside a ``<details>`` toggle so the four foot links
|
| 78 |
+
(Tasks, Submit, Contribute, Method) stay visible without scrolling. The toggle
|
| 79 |
+
is pure HTML/CSS, keeping navigation reload-based and JavaScript-free.
|
| 80 |
|
| 81 |
``active_slug`` highlights the board a visitor is on; ``active_tab`` highlights
|
| 82 |
+
a foot link, and its absence means we are on a board or the overview, so the
|
| 83 |
+
"Boards" menu is the active one. Open boards link to Contribute, mirroring the
|
| 84 |
+
overview cards.
|
| 85 |
"""
|
| 86 |
+
on_boards = active_tab is None
|
| 87 |
+
summary_cls = "pm-rail-summary pm-active" if on_boards else "pm-rail-summary"
|
| 88 |
+
parts = [
|
| 89 |
+
'<div class="pm-rail-inner">',
|
| 90 |
+
BRAND,
|
| 91 |
+
'<details class="pm-rail-section" open>',
|
| 92 |
+
f'<summary class="{summary_cls}"><span>Boards</span>{CHEVRON}</summary>',
|
| 93 |
+
'<div class="pm-rail-sections">',
|
| 94 |
+
]
|
| 95 |
for group in SECTIONS:
|
| 96 |
cards, opens = in_group(boards, group), open_in_group(boards, group)
|
| 97 |
if not cards and not opens:
|
|
|
|
| 113 |
'<span class="pm-badge pm-badge--quiet">open</span></a>'
|
| 114 |
)
|
| 115 |
parts.append("</div>")
|
| 116 |
+
parts.append("</div></details>")
|
| 117 |
parts.append('<div class="pm-rail-foot">')
|
| 118 |
for tab, label in FOOT_LINKS:
|
| 119 |
cls = "pm-foot-link pm-active" if tab == active_tab else "pm-foot-link"
|