Spaces:
Running on Zero
Running on Zero
Centauri Cognitive Simulator: interactive Psych-101 experiments with trial-by-trial human-choice prediction
Browse files- README.md +55 -5
- app.py +986 -0
- psych101_examples.json +38 -0
- requirements.txt +5 -0
README.md
CHANGED
|
@@ -1,13 +1,63 @@
|
|
| 1 |
---
|
| 2 |
title: Centauri Cognitive Simulator
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.22.0
|
| 8 |
-
python_version: '3.12'
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
title: Centauri Cognitive Simulator
|
| 3 |
+
emoji: 🧠
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: green
|
| 6 |
sdk: gradio
|
| 7 |
sdk_version: 6.22.0
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
short_description: Small cognition models predict your choices, trial by trial
|
| 12 |
+
python_version: "3.12"
|
| 13 |
+
startup_duration_timeout: 1h
|
| 14 |
+
models:
|
| 15 |
+
- socius/Qwentaur-8B-LoRA-r16
|
| 16 |
+
- socius/Qwentaur-0.6B-LoRA-r16
|
| 17 |
+
datasets:
|
| 18 |
+
- marcelbinz/Psych-101
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# 🧠 Centauri Cognitive Simulator
|
| 22 |
+
|
| 23 |
+
Play a real psychology experiment while three models predict, trial by trial, what a
|
| 24 |
+
**human** would do next:
|
| 25 |
+
|
| 26 |
+
| predictor | what it is |
|
| 27 |
+
|---|---|
|
| 28 |
+
| **Qwentaur-8B** | [`socius/Qwentaur-8B-LoRA-r16`](https://huggingface.co/socius/Qwentaur-8B-LoRA-r16) — LoRA r=16 on `unsloth/Qwen3-8B-Base` |
|
| 29 |
+
| **Qwentaur-0.6B** | [`socius/Qwentaur-0.6B-LoRA-r16`](https://huggingface.co/socius/Qwentaur-0.6B-LoRA-r16) — 13× smaller |
|
| 30 |
+
| **Qwen3-8B-Base** | the same 8B weights with the adapter switched off (no fine-tuning) |
|
| 31 |
+
|
| 32 |
+
From *Small Foundation Models of Human Cognition and Behaviour* (Oh & Gobet, 2026) —
|
| 33 |
+
[paper](https://huggingface.co/papers/2608.05224) ·
|
| 34 |
+
[code](https://github.com/socius-org/Centauri). The paper's headline claim is that
|
| 35 |
+
sub-billion-parameter models already match a 70B Centaur at fitting human choices
|
| 36 |
+
in-distribution; the side-by-side bars and the running loss table let you watch that.
|
| 37 |
+
|
| 38 |
+
## How it works
|
| 39 |
+
|
| 40 |
+
Everything the models see is plain [Psych-101](https://huggingface.co/datasets/marcelbinz/Psych-101)
|
| 41 |
+
text, exactly as during fine-tuning: human choices are wrapped in `<<...>>`, and the
|
| 42 |
+
choice distribution is read straight off the next-token logits after `You press <<`,
|
| 43 |
+
renormalised over the available option letters. No sampling, no prompt engineering,
|
| 44 |
+
no chat template.
|
| 45 |
+
|
| 46 |
+
Four experiments are implemented with the dataset's verbatim instruction wording:
|
| 47 |
+
|
| 48 |
+
- **Horizon task** (Wilson et al., 2014) — explore or exploit two slot machines
|
| 49 |
+
- **Iowa Gambling Task** (Steingroever et al., 2015) — learning under risk
|
| 50 |
+
- **Intertemporal choice** (Ruggeri et al., 2022) — patience
|
| 51 |
+
- **Multi-attribute inference** (Hilbig & Moshagen, 2014) — cue integration
|
| 52 |
+
|
| 53 |
+
The second tab scores the models the way the paper does — mean negative log-likelihood
|
| 54 |
+
of a **real participant's** actual choices — on human sessions bundled verbatim from
|
| 55 |
+
Psych-101 (Binz et al., 2025, Apache-2.0).
|
| 56 |
+
|
| 57 |
+
## Credits
|
| 58 |
+
|
| 59 |
+
- Adapters: `socius/*` (Apache-2.0), Oh & Gobet, *Small Foundation Models of Human
|
| 60 |
+
Cognition and Behaviour*, 2026.
|
| 61 |
+
- Base models: `unsloth/Qwen3-8B-Base`, `unsloth/Qwen3-0.6B-Base`.
|
| 62 |
+
- Task wording and bundled human transcripts: [Psych-101](https://huggingface.co/datasets/marcelbinz/Psych-101)
|
| 63 |
+
(Binz et al., 2025, Apache-2.0).
|
app.py
ADDED
|
@@ -0,0 +1,986 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Centauri Cognitive Simulator.
|
| 2 |
+
|
| 3 |
+
Play a psychology experiment trial by trial while small foundation models of human
|
| 4 |
+
cognition (Centauri / Qwentaur LoRA adapters over Qwen3-Base, Oh & Gobet 2026)
|
| 5 |
+
predict what a *human* would do next.
|
| 6 |
+
|
| 7 |
+
Everything the models see is plain Psych-101-formatted text, exactly as in training:
|
| 8 |
+
choices are wrapped in `<<...>>` and the model's choice distribution is read off the
|
| 9 |
+
next-token logits right after `You press <<`.
|
| 10 |
+
"""
|
| 11 |
+
|
| 12 |
+
import os
|
| 13 |
+
|
| 14 |
+
os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
|
| 15 |
+
|
| 16 |
+
import spaces # noqa: E402 (must precede torch / transformers)
|
| 17 |
+
|
| 18 |
+
import html
|
| 19 |
+
import json
|
| 20 |
+
import math
|
| 21 |
+
import random
|
| 22 |
+
import re
|
| 23 |
+
import time
|
| 24 |
+
from typing import Any, Dict, List, Optional, Tuple
|
| 25 |
+
|
| 26 |
+
import gradio as gr
|
| 27 |
+
import torch
|
| 28 |
+
from peft import PeftModel
|
| 29 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 30 |
+
|
| 31 |
+
# --------------------------------------------------------------------------------------
|
| 32 |
+
# Models
|
| 33 |
+
# --------------------------------------------------------------------------------------
|
| 34 |
+
|
| 35 |
+
PRIMARY = "socius/Qwentaur-8B-LoRA-r16"
|
| 36 |
+
SMALL = "socius/Qwentaur-0.6B-LoRA-r16"
|
| 37 |
+
BASE_8B = "unsloth/Qwen3-8B-Base"
|
| 38 |
+
BASE_06B = "unsloth/Qwen3-0.6B-Base"
|
| 39 |
+
|
| 40 |
+
# key -> display metadata. "big"/"small" are the fine-tuned adapters, "raw" is the
|
| 41 |
+
# 8B base model with the adapter switched off (free — same weights in VRAM).
|
| 42 |
+
PREDICTORS = [
|
| 43 |
+
("big", "Qwentaur-8B", "#4f46e5", "LoRA r=16 on Qwen3-8B-Base"),
|
| 44 |
+
("small", "Qwentaur-0.6B", "#0d9488", "LoRA r=16 on Qwen3-0.6B-Base"),
|
| 45 |
+
("raw", "Qwen3-8B-Base", "#a1a1aa", "no fine-tuning (adapter off)"),
|
| 46 |
+
]
|
| 47 |
+
PRED_LABEL = {k: label for k, label, _, _ in PREDICTORS}
|
| 48 |
+
PRED_COLOR = {k: color for k, _, color, _ in PREDICTORS}
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _load(base_id: str, adapter_id: str):
|
| 52 |
+
tok = AutoTokenizer.from_pretrained(adapter_id)
|
| 53 |
+
model = AutoModelForCausalLM.from_pretrained(base_id, dtype=torch.bfloat16)
|
| 54 |
+
model = PeftModel.from_pretrained(model, adapter_id)
|
| 55 |
+
model.eval()
|
| 56 |
+
model = model.to("cuda")
|
| 57 |
+
return tok, model
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
print("Loading Qwentaur-8B (Qwen3-8B-Base + LoRA r=16) ...", flush=True)
|
| 61 |
+
TOK_BIG, MODEL_BIG = _load(BASE_8B, PRIMARY)
|
| 62 |
+
print("Loading Qwentaur-0.6B (Qwen3-0.6B-Base + LoRA r=16) ...", flush=True)
|
| 63 |
+
TOK_SMALL, MODEL_SMALL = _load(BASE_06B, SMALL)
|
| 64 |
+
print("Models ready.", flush=True)
|
| 65 |
+
|
| 66 |
+
# Single-token uppercase letters, shared by both (identical Qwen3) tokenizers.
|
| 67 |
+
LETTER_POOL = [
|
| 68 |
+
c
|
| 69 |
+
for c in "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
| 70 |
+
if len(TOK_BIG(c, add_special_tokens=False).input_ids) == 1
|
| 71 |
+
and len(TOK_SMALL(c, add_special_tokens=False).input_ids) == 1
|
| 72 |
+
]
|
| 73 |
+
print(f"{len(LETTER_POOL)} single-token option letters available.", flush=True)
|
| 74 |
+
|
| 75 |
+
MAX_SCORE_TOKENS = 3072
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
@torch.no_grad()
|
| 79 |
+
def _next_letter_probs(model, tok, prefix: str, letters: List[str]) -> Dict[str, float]:
|
| 80 |
+
"""Renormalised P(letter) read off the next-token logits after `... You press <<`."""
|
| 81 |
+
ids = tok(prefix, return_tensors="pt", add_special_tokens=False).input_ids.to("cuda")
|
| 82 |
+
if ids.shape[1] > MAX_SCORE_TOKENS:
|
| 83 |
+
ids = ids[:, -MAX_SCORE_TOKENS:]
|
| 84 |
+
logits = model(ids).logits[0, -1].float()
|
| 85 |
+
logp = torch.log_softmax(logits, dim=-1)
|
| 86 |
+
raw = {}
|
| 87 |
+
for letter in letters:
|
| 88 |
+
tid = tok(letter, add_special_tokens=False).input_ids[0]
|
| 89 |
+
raw[letter] = float(logp[tid])
|
| 90 |
+
mx = max(raw.values())
|
| 91 |
+
ex = {k: math.exp(v - mx) for k, v in raw.items()}
|
| 92 |
+
z = sum(ex.values())
|
| 93 |
+
return {k: v / z for k, v in ex.items()}
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def _predict_all(prefix: str, letters: List[str]) -> Dict[str, Dict[str, float]]:
|
| 97 |
+
"""Choice distributions from all three predictors for one upcoming choice."""
|
| 98 |
+
out = {"big": _next_letter_probs(MODEL_BIG, TOK_BIG, prefix, letters)}
|
| 99 |
+
with MODEL_BIG.disable_adapter():
|
| 100 |
+
out["raw"] = _next_letter_probs(MODEL_BIG, TOK_BIG, prefix, letters)
|
| 101 |
+
out["small"] = _next_letter_probs(MODEL_SMALL, TOK_SMALL, prefix, letters)
|
| 102 |
+
return out
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
# --------------------------------------------------------------------------------------
|
| 106 |
+
# Psych-101 task environments
|
| 107 |
+
#
|
| 108 |
+
# Wording is copied verbatim from the Psych-101 transcripts (Binz et al.) so the models
|
| 109 |
+
# see text drawn from exactly the distribution they were fine-tuned on. Only the option
|
| 110 |
+
# letters are re-randomised per session, as in the original dataset.
|
| 111 |
+
# --------------------------------------------------------------------------------------
|
| 112 |
+
|
| 113 |
+
TASK_ORDER = ["bandit", "igt", "itc", "cue"]
|
| 114 |
+
TASK_LABELS = {
|
| 115 |
+
"bandit": "🎰 Horizon task — explore or exploit (Wilson et al., 2014)",
|
| 116 |
+
"igt": "🃏 Iowa Gambling Task — learning under risk (Steingroever et al., 2015)",
|
| 117 |
+
"itc": "⏳ Intertemporal choice — patience (Ruggeri et al., 2022)",
|
| 118 |
+
"cue": "🔍 Multi-attribute inference — cue integration (Hilbig & Moshagen, 2014)",
|
| 119 |
+
}
|
| 120 |
+
LABEL_TO_TASK = {v: k for k, v in TASK_LABELS.items()}
|
| 121 |
+
|
| 122 |
+
IGT_DECKS = {
|
| 123 |
+
# (win, [loss schedule over a block of 10 cards]) — classic Bechara payoffs
|
| 124 |
+
"bad_freq": (100.0, [0, 0, 150, 0, 300, 0, 200, 0, 250, 350]),
|
| 125 |
+
"bad_rare": (100.0, [0, 0, 0, 0, 0, 0, 0, 0, 0, 1250]),
|
| 126 |
+
"good_freq": (50.0, [0, 0, 50, 0, 50, 0, 50, 0, 50, 50]),
|
| 127 |
+
"good_rare": (50.0, [0, 0, 0, 0, 0, 0, 0, 0, 0, 250]),
|
| 128 |
+
}
|
| 129 |
+
|
| 130 |
+
# The ten trials of the Ruggeri et al. intertemporal-choice block, verbatim.
|
| 131 |
+
ITC_TRIALS = [
|
| 132 |
+
("receiving 500$ immediately", "receiving 550$ in one year"),
|
| 133 |
+
("receiving 500$ immediately", "receiving 600$ in one year"),
|
| 134 |
+
("receiving 500$ immediately", "receiving 750$ in one year"),
|
| 135 |
+
("paying 500$ immediately", "paying 550$ in one year"),
|
| 136 |
+
("paying 500$ immediately", "paying 510$ in one year"),
|
| 137 |
+
("paying 500$ immediately", "paying 505$ in one year"),
|
| 138 |
+
("receiving 5000$ immediately", "receiving 5500$ in one year"),
|
| 139 |
+
("receiving 5000$ immediately", "receiving 6000$ in one year"),
|
| 140 |
+
("receiving 5000$ immediately", "receiving 7500$ in one year"),
|
| 141 |
+
("receiving 500$ in one year", "receiving 750$ in two years"),
|
| 142 |
+
]
|
| 143 |
+
|
| 144 |
+
CUE_VALIDITIES = [0.9, 0.8, 0.7, 0.6]
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
def new_session(task: str, seed: int, blind: bool) -> Dict[str, Any]:
|
| 148 |
+
rng = random.Random(seed)
|
| 149 |
+
st: Dict[str, Any] = {
|
| 150 |
+
"task": task,
|
| 151 |
+
"seed": int(seed),
|
| 152 |
+
"blind": bool(blind),
|
| 153 |
+
"trial": 0,
|
| 154 |
+
"log": [],
|
| 155 |
+
"pred": {},
|
| 156 |
+
"done": False,
|
| 157 |
+
}
|
| 158 |
+
|
| 159 |
+
if task == "bandit":
|
| 160 |
+
letters = rng.sample(LETTER_POOL, 2)
|
| 161 |
+
st["letters"] = letters
|
| 162 |
+
low = rng.randint(30, 58)
|
| 163 |
+
delta = rng.choice([4, 8, 12, 20, 30])
|
| 164 |
+
means = [low, low + delta]
|
| 165 |
+
rng.shuffle(means)
|
| 166 |
+
# Pre-generate the full (counterfactual) reward table for 4 instructed + 6 free trials.
|
| 167 |
+
rewards = [
|
| 168 |
+
[max(1, min(99, int(round(rng.gauss(m, 8))))) for _ in range(10)] for m in means
|
| 169 |
+
]
|
| 170 |
+
forced = ([0] * 3 + [1]) if rng.random() < 0.5 else [0, 0, 1, 1]
|
| 171 |
+
rng.shuffle(forced)
|
| 172 |
+
st["plan"] = {"rewards": rewards, "forced": forced, "means": means}
|
| 173 |
+
st["n_trials"] = 6
|
| 174 |
+
st["text"] = (
|
| 175 |
+
f"You are participating in multiple games involving two slot machines, labeled {letters[0]} and {letters[1]}.\n"
|
| 176 |
+
"The two slot machines are different across different games.\n"
|
| 177 |
+
"Each time you choose a slot machine, you get some points.\n"
|
| 178 |
+
"You choose a slot machine by pressing the corresponding key.\n"
|
| 179 |
+
"Each slot machine tends to pay out about the same amount of points on average.\n"
|
| 180 |
+
"Your goal is to choose the slot machines that will give you the most points across the experiment.\n"
|
| 181 |
+
"The first 4 trials in each game are instructed trials where you will be told which slot machine to choose.\n"
|
| 182 |
+
"After these instructed trials, you will have the freedom to choose for either 1 or 6 trials.\n"
|
| 183 |
+
"\nGame 1. There are 10 trials in this game.\n"
|
| 184 |
+
)
|
| 185 |
+
counts = [0, 0]
|
| 186 |
+
observed: List[List[int]] = [[], []]
|
| 187 |
+
for arm in forced:
|
| 188 |
+
r = rewards[arm][counts[arm]]
|
| 189 |
+
counts[arm] += 1
|
| 190 |
+
observed[arm].append(r)
|
| 191 |
+
st["text"] += f"You are instructed to press {letters[arm]} and get {r} points.\n"
|
| 192 |
+
st["counts"] = counts
|
| 193 |
+
st["observed"] = observed
|
| 194 |
+
st["total"] = sum(sum(o) for o in observed)
|
| 195 |
+
|
| 196 |
+
elif task == "igt":
|
| 197 |
+
letters = rng.sample(LETTER_POOL, 4)
|
| 198 |
+
st["letters"] = letters
|
| 199 |
+
kinds = list(IGT_DECKS.keys())
|
| 200 |
+
rng.shuffle(kinds)
|
| 201 |
+
schedule = {}
|
| 202 |
+
for letter, kind in zip(letters, kinds):
|
| 203 |
+
win, losses = IGT_DECKS[kind]
|
| 204 |
+
cards = []
|
| 205 |
+
for _ in range(4): # 40 cards per deck is plenty for 30 trials
|
| 206 |
+
block = list(losses)
|
| 207 |
+
rng.shuffle(block)
|
| 208 |
+
cards += [(win, float(x)) for x in block]
|
| 209 |
+
schedule[letter] = cards
|
| 210 |
+
st["plan"] = {"schedule": schedule, "kinds": dict(zip(letters, kinds))}
|
| 211 |
+
st["n_trials"] = 30
|
| 212 |
+
st["counts"] = {letter: 0 for letter in letters}
|
| 213 |
+
st["net"] = {letter: 0.0 for letter in letters}
|
| 214 |
+
st["balance"] = 2000.0
|
| 215 |
+
st["text"] = (
|
| 216 |
+
f"You see in front of you four decks of cards labeled {letters[0]}, {letters[1]}, {letters[2]}, and {letters[3]}.\n"
|
| 217 |
+
"You get a loan of 2000$ of play money.\n"
|
| 218 |
+
"You have to select one card at a time, from any of the four decks, for 100 trials.\n"
|
| 219 |
+
"You select a card from a deck by pressing the corresponding key.\n"
|
| 220 |
+
"After turning a card, you win some money, the amount varies with the deck.\n"
|
| 221 |
+
"You sometimes also have to pay a penalty, which also varies with the deck.\n"
|
| 222 |
+
"Your goal is to maximize profit on the loan of the play money.\n\n"
|
| 223 |
+
)
|
| 224 |
+
|
| 225 |
+
elif task == "itc":
|
| 226 |
+
letters = rng.sample(LETTER_POOL, 2)
|
| 227 |
+
st["letters"] = letters
|
| 228 |
+
st["plan"] = {"trials": ITC_TRIALS}
|
| 229 |
+
st["n_trials"] = len(ITC_TRIALS)
|
| 230 |
+
st["patient"] = 0
|
| 231 |
+
st["text"] = (
|
| 232 |
+
f"In the following you will be presented with multiple choices between two options {letters[0]} and {letters[1]}.\n"
|
| 233 |
+
"Please name which option you would prefer by pressing the corresponding key.\n\n"
|
| 234 |
+
)
|
| 235 |
+
|
| 236 |
+
elif task == "cue":
|
| 237 |
+
letters = rng.sample(LETTER_POOL, 2)
|
| 238 |
+
st["letters"] = letters
|
| 239 |
+
stimuli = []
|
| 240 |
+
while len(stimuli) < 20:
|
| 241 |
+
a = [rng.randint(0, 1) for _ in range(4)]
|
| 242 |
+
b = [rng.randint(0, 1) for _ in range(4)]
|
| 243 |
+
if a != b:
|
| 244 |
+
stimuli.append([a, b])
|
| 245 |
+
st["plan"] = {"stimuli": stimuli}
|
| 246 |
+
st["n_trials"] = 20
|
| 247 |
+
st["optimal"] = 0
|
| 248 |
+
st["text"] = (
|
| 249 |
+
f"You are repeatedly presented with two options, labeled {letters[0]} and {letters[1]}.\n"
|
| 250 |
+
"Each option represents a fictitious product and you have to infer which product is superior in terms of quality.\n"
|
| 251 |
+
"You select a product by pressing the corresponding key.\n"
|
| 252 |
+
"For each decision, you are provided with four expert ratings (with 1 representing a positive and 0 representing a negative rating).\n"
|
| 253 |
+
"The four experts differ in their validity.\n"
|
| 254 |
+
"The ratings of experts are given in descending order of their validity (having validities of 90%, 80%, 70%, and 60%).\n\n"
|
| 255 |
+
)
|
| 256 |
+
else:
|
| 257 |
+
raise ValueError(task)
|
| 258 |
+
|
| 259 |
+
return st
|
| 260 |
+
|
| 261 |
+
|
| 262 |
+
def pre_text(st: Dict[str, Any]) -> str:
|
| 263 |
+
"""Text of the current trial up to (not including) ` <<`."""
|
| 264 |
+
task, letters, t = st["task"], st["letters"], st["trial"]
|
| 265 |
+
if task in ("bandit", "igt"):
|
| 266 |
+
return "You press"
|
| 267 |
+
if task == "itc":
|
| 268 |
+
a, b = st["plan"]["trials"][t]
|
| 269 |
+
return (
|
| 270 |
+
f"You have the choice between {a} (press {letters[0]}) or "
|
| 271 |
+
f"{b} (press {letters[1]}). You press"
|
| 272 |
+
)
|
| 273 |
+
if task == "cue":
|
| 274 |
+
a, b = st["plan"]["stimuli"][t]
|
| 275 |
+
fmt = lambda v: "[" + " ".join(str(x) for x in v) + "]" # noqa: E731
|
| 276 |
+
return (
|
| 277 |
+
f"Product {letters[0]} ratings: {fmt(a)}. "
|
| 278 |
+
f"Product {letters[1]} ratings: {fmt(b)}. You press"
|
| 279 |
+
)
|
| 280 |
+
raise ValueError(task)
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def score_prefix(st: Dict[str, Any]) -> str:
|
| 284 |
+
return st["text"] + pre_text(st) + " <<"
|
| 285 |
+
|
| 286 |
+
|
| 287 |
+
def apply_choice(st: Dict[str, Any], letter: str) -> str:
|
| 288 |
+
"""Append the completed trial line to the transcript; returns a plain-text outcome."""
|
| 289 |
+
task, letters, t = st["task"], st["letters"], st["trial"]
|
| 290 |
+
line = pre_text(st) + f" <<{letter}>>"
|
| 291 |
+
|
| 292 |
+
if task == "bandit":
|
| 293 |
+
arm = letters.index(letter)
|
| 294 |
+
r = st["plan"]["rewards"][arm][st["counts"][arm]]
|
| 295 |
+
st["counts"][arm] += 1
|
| 296 |
+
st["observed"][arm].append(r)
|
| 297 |
+
st["total"] += r
|
| 298 |
+
line += f" and get {r} points.\n"
|
| 299 |
+
outcome = f"Machine {letter} paid {r} points."
|
| 300 |
+
elif task == "igt":
|
| 301 |
+
win, loss = st["plan"]["schedule"][letter][st["counts"][letter]]
|
| 302 |
+
st["counts"][letter] += 1
|
| 303 |
+
st["net"][letter] += win - loss
|
| 304 |
+
st["balance"] += win - loss
|
| 305 |
+
line += f". You win {win:.1f}$ and lose {loss:.1f}$.\n"
|
| 306 |
+
outcome = f"Deck {letter}: won ${win:.0f}" + (f", lost ${loss:.0f}" if loss else ", no penalty")
|
| 307 |
+
elif task == "itc":
|
| 308 |
+
if letter == letters[1]:
|
| 309 |
+
st["patient"] += 1
|
| 310 |
+
line += ".\n"
|
| 311 |
+
outcome = f"You chose option {letter}."
|
| 312 |
+
elif task == "cue":
|
| 313 |
+
a, b = st["plan"]["stimuli"][t]
|
| 314 |
+
wa = sum(v * math.log(p / (1 - p)) for v, p in zip(a, CUE_VALIDITIES))
|
| 315 |
+
wb = sum(v * math.log(p / (1 - p)) for v, p in zip(b, CUE_VALIDITIES))
|
| 316 |
+
best = letters[0] if wa >= wb else letters[1]
|
| 317 |
+
if letter == best:
|
| 318 |
+
st["optimal"] += 1
|
| 319 |
+
line += ".\n"
|
| 320 |
+
outcome = (
|
| 321 |
+
f"Optimal cue integration would pick {best}."
|
| 322 |
+
if letter != best
|
| 323 |
+
else f"{letter} is also what optimal cue integration picks."
|
| 324 |
+
)
|
| 325 |
+
else:
|
| 326 |
+
raise ValueError(task)
|
| 327 |
+
|
| 328 |
+
st["text"] += line
|
| 329 |
+
st["trial"] += 1
|
| 330 |
+
return outcome
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
# --------------------------------------------------------------------------------------
|
| 334 |
+
# Rendering
|
| 335 |
+
# --------------------------------------------------------------------------------------
|
| 336 |
+
|
| 337 |
+
CSS_BLOCK = """
|
| 338 |
+
<style>
|
| 339 |
+
.cs-wrap{font-family:var(--font,ui-sans-serif,system-ui);}
|
| 340 |
+
.cs-head{font-size:13px;opacity:.75;margin:0 0 8px 2px;letter-spacing:.02em}
|
| 341 |
+
.cs-cards{display:flex;gap:10px;flex-wrap:wrap}
|
| 342 |
+
.cs-card{flex:1 1 130px;min-width:120px;border:1px solid var(--border-color-primary,#e5e7eb);
|
| 343 |
+
border-radius:12px;padding:10px 12px;background:var(--background-fill-secondary,#fafafa)}
|
| 344 |
+
.cs-card.sel{border-color:#f59e0b;box-shadow:0 0 0 2px rgba(245,158,11,.25)}
|
| 345 |
+
.cs-key{font-size:26px;font-weight:700;line-height:1.1}
|
| 346 |
+
.cs-sub{font-size:12px;opacity:.72;margin-top:3px;line-height:1.45}
|
| 347 |
+
.cs-out{margin-top:10px;font-size:13px;padding:7px 10px;border-radius:8px;
|
| 348 |
+
background:rgba(245,158,11,.12);border:1px solid rgba(245,158,11,.35)}
|
| 349 |
+
.cs-mrow{margin:0 0 12px 0}
|
| 350 |
+
.cs-mname{font-size:12px;font-weight:600;margin-bottom:4px;display:flex;
|
| 351 |
+
justify-content:space-between;align-items:baseline;gap:8px}
|
| 352 |
+
.cs-mnote{font-weight:400;opacity:.6;font-size:11px}
|
| 353 |
+
.cs-bar{display:flex;align-items:center;gap:6px;margin:2px 0;font-size:11px}
|
| 354 |
+
.cs-bl{width:16px;text-align:right;opacity:.8;font-weight:600}
|
| 355 |
+
.cs-btrack{flex:1;height:14px;border-radius:7px;background:var(--background-fill-secondary,#eee);overflow:hidden}
|
| 356 |
+
.cs-bfill{height:100%;border-radius:7px}
|
| 357 |
+
.cs-bv{width:42px;font-variant-numeric:tabular-nums;opacity:.8}
|
| 358 |
+
.cs-pick{color:#f59e0b;font-weight:700}
|
| 359 |
+
.cs-tbl{width:100%;border-collapse:collapse;font-size:12px;margin-top:4px}
|
| 360 |
+
.cs-tbl th,.cs-tbl td{padding:4px 6px;text-align:right;border-bottom:1px solid var(--border-color-primary,#eee)}
|
| 361 |
+
.cs-tbl th:first-child,.cs-tbl td:first-child{text-align:left}
|
| 362 |
+
.cs-note{font-size:11.5px;opacity:.65;margin-top:8px;line-height:1.5}
|
| 363 |
+
.cs-tx{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12px;line-height:1.75;
|
| 364 |
+
white-space:pre-wrap;max-height:420px;overflow:auto;padding:10px;border-radius:8px;
|
| 365 |
+
border:1px solid var(--border-color-primary,#e5e7eb)}
|
| 366 |
+
.cs-ch{border-radius:4px;padding:1px 3px;font-weight:700}
|
| 367 |
+
</style>
|
| 368 |
+
"""
|
| 369 |
+
|
| 370 |
+
|
| 371 |
+
def _pips(vals: List[int]) -> str:
|
| 372 |
+
out = []
|
| 373 |
+
for v, p in zip(vals, CUE_VALIDITIES):
|
| 374 |
+
col = "#16a34a" if v else "#d4d4d8"
|
| 375 |
+
out.append(
|
| 376 |
+
f"<span title='validity {int(p*100)}%' style='display:inline-block;width:13px;height:13px;"
|
| 377 |
+
f"border-radius:3px;background:{col};margin-right:3px'></span>"
|
| 378 |
+
)
|
| 379 |
+
return "".join(out)
|
| 380 |
+
|
| 381 |
+
|
| 382 |
+
def render_board(st: Optional[Dict[str, Any]], outcome: str = "") -> str:
|
| 383 |
+
if st is None:
|
| 384 |
+
return (
|
| 385 |
+
CSS_BLOCK
|
| 386 |
+
+ "<div class='cs-wrap'><div class='cs-head'>No session yet</div>"
|
| 387 |
+
"<div class='cs-sub'>Pick an experiment above and press <b>Start experiment</b>.</div></div>"
|
| 388 |
+
)
|
| 389 |
+
|
| 390 |
+
task, letters, t, n = st["task"], st["letters"], st["trial"], st["n_trials"]
|
| 391 |
+
done = st["trial"] >= n
|
| 392 |
+
cards = []
|
| 393 |
+
|
| 394 |
+
if task == "bandit":
|
| 395 |
+
head = (
|
| 396 |
+
f"Game 1 · free choice {min(t + 1, n)} of {n} · <b>{st['total']} points</b> so far"
|
| 397 |
+
if not done
|
| 398 |
+
else f"Game over · <b>{st['total']} points</b>"
|
| 399 |
+
)
|
| 400 |
+
for i, letter in enumerate(letters):
|
| 401 |
+
obs = st["observed"][i]
|
| 402 |
+
seen = ", ".join(str(x) for x in obs) if obs else "never played"
|
| 403 |
+
avg = f"mean {sum(obs)/len(obs):.1f}" if obs else "unknown"
|
| 404 |
+
cards.append((letter, f"slot machine {letter}", [f"played {st['counts'][i]}×", seen, avg]))
|
| 405 |
+
elif task == "igt":
|
| 406 |
+
head = (
|
| 407 |
+
f"Card {min(t + 1, n)} of {n} · balance <b>${st['balance']:.0f}</b>"
|
| 408 |
+
if not done
|
| 409 |
+
else f"Finished · balance <b>${st['balance']:.0f}</b>"
|
| 410 |
+
)
|
| 411 |
+
for letter in letters:
|
| 412 |
+
cards.append(
|
| 413 |
+
(
|
| 414 |
+
letter,
|
| 415 |
+
f"deck {letter}",
|
| 416 |
+
[f"played {st['counts'][letter]}×", f"net ${st['net'][letter]:+.0f}"],
|
| 417 |
+
)
|
| 418 |
+
)
|
| 419 |
+
elif task == "itc":
|
| 420 |
+
head = f"Choice {min(t + 1, n)} of {n} · you took the later option {st['patient']}×"
|
| 421 |
+
if done:
|
| 422 |
+
head = f"Finished · you took the later option {st['patient']}/{n} times"
|
| 423 |
+
trial = st["plan"]["trials"][min(t, n - 1)]
|
| 424 |
+
for j, (letter, opt) in enumerate(zip(letters, trial)):
|
| 425 |
+
cards.append((letter, opt, ["the sooner option" if j == 0 else "the later option"]))
|
| 426 |
+
else: # cue
|
| 427 |
+
head = f"Decision {min(t + 1, n)} of {n} · matched optimal integration {st['optimal']}×"
|
| 428 |
+
if done:
|
| 429 |
+
head = f"Finished · matched optimal integration {st['optimal']}/{n} times"
|
| 430 |
+
stim = st["plan"]["stimuli"][min(t, n - 1)]
|
| 431 |
+
for letter, vals in zip(letters, stim):
|
| 432 |
+
cards.append((letter, f"product {letter}", [_pips(vals), "experts 90/80/70/60%"]))
|
| 433 |
+
|
| 434 |
+
last_pick = st["log"][-1]["choice"] if st["log"] else None
|
| 435 |
+
body = []
|
| 436 |
+
for letter, title, lines in cards:
|
| 437 |
+
sel = " sel" if letter == last_pick else ""
|
| 438 |
+
subs = "<br>".join(lines)
|
| 439 |
+
body.append(
|
| 440 |
+
f"<div class='cs-card{sel}'><div class='cs-key'>{html.escape(letter)}</div>"
|
| 441 |
+
f"<div class='cs-sub'>{title}</div><div class='cs-sub'>{subs}</div></div>"
|
| 442 |
+
)
|
| 443 |
+
|
| 444 |
+
out = f"<div class='cs-out'>{html.escape(outcome)}</div>" if outcome else ""
|
| 445 |
+
return (
|
| 446 |
+
CSS_BLOCK
|
| 447 |
+
+ f"<div class='cs-wrap'><div class='cs-head'>{head}</div>"
|
| 448 |
+
+ f"<div class='cs-cards'>{''.join(body)}</div>{out}</div>"
|
| 449 |
+
)
|
| 450 |
+
|
| 451 |
+
|
| 452 |
+
def _bars(probs: Dict[str, float], color: str, letters: List[str], pick: Optional[str]) -> str:
|
| 453 |
+
rows = []
|
| 454 |
+
top = max(probs, key=probs.get)
|
| 455 |
+
for letter in letters:
|
| 456 |
+
p = probs[letter]
|
| 457 |
+
mark = " <span class='cs-pick'>← you</span>" if letter == pick else ""
|
| 458 |
+
star = "★" if letter == top else ""
|
| 459 |
+
rows.append(
|
| 460 |
+
f"<div class='cs-bar'><div class='cs-bl'>{html.escape(letter)}</div>"
|
| 461 |
+
f"<div class='cs-btrack'><div class='cs-bfill' style='width:{p*100:.1f}%;background:{color}'></div></div>"
|
| 462 |
+
f"<div class='cs-bv'>{p*100:.1f}%</div><div>{star}{mark}</div></div>"
|
| 463 |
+
)
|
| 464 |
+
return "".join(rows)
|
| 465 |
+
|
| 466 |
+
|
| 467 |
+
def render_predictions(st: Optional[Dict[str, Any]]) -> str:
|
| 468 |
+
if st is None:
|
| 469 |
+
return (
|
| 470 |
+
CSS_BLOCK
|
| 471 |
+
+ "<div class='cs-wrap'><div class='cs-head'>Model predictions</div>"
|
| 472 |
+
"<div class='cs-sub'>Start a session to see what each model expects a human to do.</div></div>"
|
| 473 |
+
)
|
| 474 |
+
|
| 475 |
+
letters, n_opt = st["letters"], len(st["letters"])
|
| 476 |
+
done = st["trial"] >= st["n_trials"]
|
| 477 |
+
chance = 1.0 / n_opt
|
| 478 |
+
blocks = []
|
| 479 |
+
|
| 480 |
+
if st["pred"] and not done and not st["blind"]:
|
| 481 |
+
title = f"Predicted next choice · trial {st['trial'] + 1}"
|
| 482 |
+
for key, label, color, note in PREDICTORS:
|
| 483 |
+
probs = st["pred"][key]
|
| 484 |
+
blocks.append(
|
| 485 |
+
f"<div class='cs-mrow'><div class='cs-mname'><span>{label}</span>"
|
| 486 |
+
f"<span class='cs-mnote'>{note}</span></div>{_bars(probs, color, letters, None)}</div>"
|
| 487 |
+
)
|
| 488 |
+
elif st["log"]:
|
| 489 |
+
title = "Last trial · what each model expected"
|
| 490 |
+
rec = st["log"][-1]
|
| 491 |
+
for key, label, color, note in PREDICTORS:
|
| 492 |
+
probs = rec["probs"][key]
|
| 493 |
+
blocks.append(
|
| 494 |
+
f"<div class='cs-mrow'><div class='cs-mname'><span>{label}</span>"
|
| 495 |
+
f"<span class='cs-mnote'>{note}</span></div>"
|
| 496 |
+
f"{_bars(probs, color, letters, rec['choice'])}</div>"
|
| 497 |
+
)
|
| 498 |
+
else:
|
| 499 |
+
title = "Model predictions"
|
| 500 |
+
blocks.append("<div class='cs-sub'>Waiting for the first trial…</div>")
|
| 501 |
+
|
| 502 |
+
# scoreboard
|
| 503 |
+
table = ""
|
| 504 |
+
if st["log"]:
|
| 505 |
+
rows = []
|
| 506 |
+
for key, label, _color, _note in PREDICTORS:
|
| 507 |
+
nll = [-math.log(max(r["probs"][key][r["choice"]], 1e-9)) for r in st["log"]]
|
| 508 |
+
hits = sum(1 for r in st["log"] if max(r["probs"][key], key=r["probs"][key].get) == r["choice"])
|
| 509 |
+
mean_nll = sum(nll) / len(nll)
|
| 510 |
+
pr2 = 1.0 - mean_nll / math.log(n_opt)
|
| 511 |
+
rows.append(
|
| 512 |
+
f"<tr><td>{label}</td><td>{mean_nll:.3f}</td><td>{pr2*100:.0f}%</td>"
|
| 513 |
+
f"<td>{hits}/{len(st['log'])}</td></tr>"
|
| 514 |
+
)
|
| 515 |
+
rows.append(
|
| 516 |
+
f"<tr><td>random guessing</td><td>{math.log(n_opt):.3f}</td><td>0%</td>"
|
| 517 |
+
f"<td>{len(st['log'])/n_opt:.1f}/{len(st['log'])}</td></tr>"
|
| 518 |
+
)
|
| 519 |
+
table = (
|
| 520 |
+
"<table class='cs-tbl'><tr><th>predictor</th><th>loss</th><th>pseudo-R²</th>"
|
| 521 |
+
f"<th>top-1 hits</th></tr>{''.join(rows)}</table>"
|
| 522 |
+
"<div class='cs-note'>loss = mean negative log-likelihood of <i>your</i> choices "
|
| 523 |
+
f"(nats/choice; {math.log(n_opt):.3f} = chance with {n_opt} options). "
|
| 524 |
+
"This is the metric the paper reports on Psych-101.</div>"
|
| 525 |
+
)
|
| 526 |
+
|
| 527 |
+
return (
|
| 528 |
+
CSS_BLOCK
|
| 529 |
+
+ f"<div class='cs-wrap'><div class='cs-head'>{title}</div>{''.join(blocks)}{table}"
|
| 530 |
+
+ (f"<div class='cs-note'>Chance level is {chance*100:.0f}% per option.</div>" if not st["log"] else "")
|
| 531 |
+
+ "</div>"
|
| 532 |
+
)
|
| 533 |
+
|
| 534 |
+
|
| 535 |
+
def render_chart(st: Optional[Dict[str, Any]]) -> str:
|
| 536 |
+
if st is None or not st["log"]:
|
| 537 |
+
return ""
|
| 538 |
+
n = len(st["log"])
|
| 539 |
+
n_opt = len(st["letters"])
|
| 540 |
+
W, H, PL, PR, PT, PB = 660, 190, 34, 12, 14, 26
|
| 541 |
+
iw, ih = W - PL - PR, H - PT - PB
|
| 542 |
+
|
| 543 |
+
def x(i):
|
| 544 |
+
return PL + (iw * (i / max(1, n - 1)) if n > 1 else iw / 2)
|
| 545 |
+
|
| 546 |
+
def y(p):
|
| 547 |
+
return PT + ih * (1 - p)
|
| 548 |
+
|
| 549 |
+
parts = [f"<svg viewBox='0 0 {W} {H}' width='100%' style='max-width:700px'>"]
|
| 550 |
+
for gy in (0.0, 0.25, 0.5, 0.75, 1.0):
|
| 551 |
+
parts.append(
|
| 552 |
+
f"<line x1='{PL}' y1='{y(gy):.1f}' x2='{W-PR}' y2='{y(gy):.1f}' "
|
| 553 |
+
f"stroke='currentColor' stroke-opacity='.12'/>"
|
| 554 |
+
f"<text x='{PL-6}' y='{y(gy)+3:.1f}' font-size='9' text-anchor='end' "
|
| 555 |
+
f"fill='currentColor' opacity='.5'>{int(gy*100)}%</text>"
|
| 556 |
+
)
|
| 557 |
+
ch = 1.0 / n_opt
|
| 558 |
+
parts.append(
|
| 559 |
+
f"<line x1='{PL}' y1='{y(ch):.1f}' x2='{W-PR}' y2='{y(ch):.1f}' stroke='#ef4444' "
|
| 560 |
+
f"stroke-dasharray='4 3' stroke-opacity='.7'/>"
|
| 561 |
+
f"<text x='{W-PR}' y='{y(ch)-4:.1f}' font-size='9' text-anchor='end' fill='#ef4444' "
|
| 562 |
+
f"opacity='.85'>chance</text>"
|
| 563 |
+
)
|
| 564 |
+
for key, label, color, _note in PREDICTORS:
|
| 565 |
+
pts = " ".join(
|
| 566 |
+
f"{x(i):.1f},{y(r['probs'][key][r['choice']]):.1f}" for i, r in enumerate(st["log"])
|
| 567 |
+
)
|
| 568 |
+
parts.append(
|
| 569 |
+
f"<polyline points='{pts}' fill='none' stroke='{color}' stroke-width='2' "
|
| 570 |
+
f"stroke-linejoin='round'/>"
|
| 571 |
+
)
|
| 572 |
+
for i, r in enumerate(st["log"]):
|
| 573 |
+
parts.append(
|
| 574 |
+
f"<circle cx='{x(i):.1f}' cy='{y(r['probs'][key][r['choice']]):.1f}' r='2.6' fill='{color}'/>"
|
| 575 |
+
)
|
| 576 |
+
parts.append(
|
| 577 |
+
f"<text x='{PL}' y='{H-8}' font-size='9.5' fill='currentColor' opacity='.6'>trial 1</text>"
|
| 578 |
+
f"<text x='{W-PR}' y='{H-8}' font-size='9.5' text-anchor='end' fill='currentColor' "
|
| 579 |
+
f"opacity='.6'>trial {n}</text></svg>"
|
| 580 |
+
)
|
| 581 |
+
legend = " ".join(
|
| 582 |
+
f"<span style='font-size:11px;margin-right:10px'>"
|
| 583 |
+
f"<span style='display:inline-block;width:9px;height:9px;border-radius:2px;"
|
| 584 |
+
f"background:{c};margin-right:4px'></span>{l}</span>"
|
| 585 |
+
for _k, l, c, _n in PREDICTORS
|
| 586 |
+
)
|
| 587 |
+
return (
|
| 588 |
+
CSS_BLOCK
|
| 589 |
+
+ "<div class='cs-wrap'><div class='cs-head'>Probability each model assigned to the choice "
|
| 590 |
+
"you actually made</div>" + "".join(parts) + f"<div>{legend}</div></div>"
|
| 591 |
+
)
|
| 592 |
+
|
| 593 |
+
|
| 594 |
+
def button_updates(st: Optional[Dict[str, Any]]) -> List[Any]:
|
| 595 |
+
ups = []
|
| 596 |
+
for i in range(4):
|
| 597 |
+
if st is None or st["trial"] >= st["n_trials"] or i >= len(st["letters"]):
|
| 598 |
+
ups.append(gr.update(visible=False))
|
| 599 |
+
else:
|
| 600 |
+
ups.append(gr.update(visible=True, value=f"Press {st['letters'][i]}"))
|
| 601 |
+
return ups
|
| 602 |
+
|
| 603 |
+
|
| 604 |
+
# --------------------------------------------------------------------------------------
|
| 605 |
+
# Interactive-session handlers (GPU)
|
| 606 |
+
# --------------------------------------------------------------------------------------
|
| 607 |
+
|
| 608 |
+
|
| 609 |
+
@spaces.GPU(duration=40)
|
| 610 |
+
def start_session(
|
| 611 |
+
task_label: str, seed: float = 0, randomize_seed: bool = True, blind: bool = False
|
| 612 |
+
) -> Tuple[Any, ...]:
|
| 613 |
+
"""Start a fresh cognitive-experiment session and predict the first choice.
|
| 614 |
+
|
| 615 |
+
Args:
|
| 616 |
+
task_label: which Psych-101 experiment to run.
|
| 617 |
+
seed: RNG seed for the experiment (stimuli, payoffs, option letters).
|
| 618 |
+
randomize_seed: draw a new random seed instead of using `seed`.
|
| 619 |
+
blind: hide the model predictions until after each trial is over.
|
| 620 |
+
"""
|
| 621 |
+
task = LABEL_TO_TASK.get(task_label, "bandit")
|
| 622 |
+
seed = random.randint(0, 2**31 - 1) if randomize_seed else int(seed)
|
| 623 |
+
st = new_session(task, seed, blind)
|
| 624 |
+
st["pred"] = _predict_all(score_prefix(st), st["letters"])
|
| 625 |
+
return (
|
| 626 |
+
st,
|
| 627 |
+
render_board(st),
|
| 628 |
+
render_predictions(st),
|
| 629 |
+
render_chart(st),
|
| 630 |
+
st["text"] + pre_text(st) + " <<",
|
| 631 |
+
gr.update(value=seed),
|
| 632 |
+
*button_updates(st),
|
| 633 |
+
)
|
| 634 |
+
|
| 635 |
+
|
| 636 |
+
@spaces.GPU(duration=40)
|
| 637 |
+
def choose(st: Optional[Dict[str, Any]], index: int = 0) -> Tuple[Any, ...]:
|
| 638 |
+
"""Register the human's choice for the current trial and predict the next one.
|
| 639 |
+
|
| 640 |
+
Args:
|
| 641 |
+
st: opaque session state.
|
| 642 |
+
index: which of the on-screen options was pressed.
|
| 643 |
+
"""
|
| 644 |
+
if st is None or st["trial"] >= st["n_trials"]:
|
| 645 |
+
return (
|
| 646 |
+
st,
|
| 647 |
+
render_board(st),
|
| 648 |
+
render_predictions(st),
|
| 649 |
+
render_chart(st),
|
| 650 |
+
"" if st is None else st["text"],
|
| 651 |
+
gr.update(),
|
| 652 |
+
*button_updates(st),
|
| 653 |
+
)
|
| 654 |
+
|
| 655 |
+
letter = st["letters"][int(index)]
|
| 656 |
+
st["log"].append({"trial": st["trial"], "choice": letter, "probs": st["pred"]})
|
| 657 |
+
outcome = apply_choice(st, letter)
|
| 658 |
+
|
| 659 |
+
if st["trial"] < st["n_trials"]:
|
| 660 |
+
st["pred"] = _predict_all(score_prefix(st), st["letters"])
|
| 661 |
+
transcript = st["text"] + pre_text(st) + " <<"
|
| 662 |
+
else:
|
| 663 |
+
st["pred"] = {}
|
| 664 |
+
st["done"] = True
|
| 665 |
+
transcript = st["text"]
|
| 666 |
+
outcome += " Session complete — start another to compare."
|
| 667 |
+
|
| 668 |
+
return (
|
| 669 |
+
st,
|
| 670 |
+
render_board(st, outcome),
|
| 671 |
+
render_predictions(st),
|
| 672 |
+
render_chart(st),
|
| 673 |
+
transcript,
|
| 674 |
+
gr.update(),
|
| 675 |
+
*button_updates(st),
|
| 676 |
+
)
|
| 677 |
+
|
| 678 |
+
|
| 679 |
+
# --------------------------------------------------------------------------------------
|
| 680 |
+
# Tab 2 — score a real Psych-101 transcript (the paper's own evaluation metric)
|
| 681 |
+
# --------------------------------------------------------------------------------------
|
| 682 |
+
|
| 683 |
+
with open("psych101_examples.json") as f:
|
| 684 |
+
EXAMPLES: Dict[str, Dict[str, str]] = json.load(f)
|
| 685 |
+
EXAMPLE_NAMES = list(EXAMPLES)
|
| 686 |
+
|
| 687 |
+
|
| 688 |
+
@torch.no_grad()
|
| 689 |
+
def _score_text(model, tok, text: str) -> Tuple[List[float], List[bool], int]:
|
| 690 |
+
"""Per-choice negative log-likelihood over every `<<...>>` response in `text`.
|
| 691 |
+
|
| 692 |
+
Mirrors the paper's evaluation: the scored span starts at the ` <<` response
|
| 693 |
+
template and ends before `>>`, and losses are summed within a response.
|
| 694 |
+
"""
|
| 695 |
+
enc = tok(text, return_tensors="pt", add_special_tokens=False, return_offsets_mapping=True)
|
| 696 |
+
ids = enc.input_ids[:, :MAX_SCORE_TOKENS].to("cuda")
|
| 697 |
+
offsets = enc.offset_mapping[0][:MAX_SCORE_TOKENS].tolist()
|
| 698 |
+
logits = model(ids).logits[0] # [T, V], bf16
|
| 699 |
+
tgt = ids[0]
|
| 700 |
+
|
| 701 |
+
# Row i-1 predicts token i. Chunked so we never materialise a full [T, vocab] fp32
|
| 702 |
+
# tensor (vocab is ~152k, so an unchunked log_softmax is gigabytes).
|
| 703 |
+
rows, targets = logits[:-1], tgt[1:]
|
| 704 |
+
lp_parts, hit_parts = [], []
|
| 705 |
+
for s in range(0, rows.shape[0], 256):
|
| 706 |
+
chunk = rows[s : s + 256].float()
|
| 707 |
+
lp = torch.log_softmax(chunk, dim=-1)
|
| 708 |
+
lp_parts.append(lp.gather(1, targets[s : s + 256, None]).squeeze(1))
|
| 709 |
+
hit_parts.append(chunk.argmax(dim=-1) == targets[s : s + 256])
|
| 710 |
+
# tok_lp[i] / argmax_hit[i] describe token i (index 0 is unpredictable -> padded).
|
| 711 |
+
tok_lp = [0.0] + torch.cat(lp_parts).tolist()
|
| 712 |
+
argmax_hit = [False] + torch.cat(hit_parts).tolist()
|
| 713 |
+
|
| 714 |
+
nlls: List[float] = []
|
| 715 |
+
hits: List[bool] = []
|
| 716 |
+
for m in re.finditer(r"<<(.*?)>>", text, flags=re.S):
|
| 717 |
+
start = m.start()
|
| 718 |
+
if start > 0 and text[start - 1] == " ":
|
| 719 |
+
start -= 1 # the ` <<` response template
|
| 720 |
+
end = m.end() - 2 # exclude `>>`
|
| 721 |
+
# every token overlapping the response span (BPE may merge across the boundary)
|
| 722 |
+
idxs = [i for i, (a, b) in enumerate(offsets) if b > a and a < end and b > start]
|
| 723 |
+
if not idxs or idxs[0] == 0:
|
| 724 |
+
continue
|
| 725 |
+
nlls.append(-sum(tok_lp[i] for i in idxs))
|
| 726 |
+
hits.append(bool(argmax_hit[idxs[-1]]))
|
| 727 |
+
used = int(ids.shape[1])
|
| 728 |
+
return nlls, hits, used
|
| 729 |
+
|
| 730 |
+
|
| 731 |
+
def _score_impl(transcript: str, colour_by: str) -> Tuple[str, str]:
|
| 732 |
+
"""Shared, undecorated scoring implementation (never call GPU-decorated fns from GPU fns)."""
|
| 733 |
+
text = (transcript or "").strip()
|
| 734 |
+
if not text or "<<" not in text:
|
| 735 |
+
return (
|
| 736 |
+
CSS_BLOCK + "<div class='cs-wrap'><div class='cs-sub'>Paste a transcript whose human "
|
| 737 |
+
"choices are wrapped in <code><< >></code>.</div></div>",
|
| 738 |
+
"",
|
| 739 |
+
)
|
| 740 |
+
|
| 741 |
+
t0 = time.perf_counter()
|
| 742 |
+
res: Dict[str, Tuple[List[float], List[bool], int]] = {}
|
| 743 |
+
res["big"] = _score_text(MODEL_BIG, TOK_BIG, text)
|
| 744 |
+
with MODEL_BIG.disable_adapter():
|
| 745 |
+
res["raw"] = _score_text(MODEL_BIG, TOK_BIG, text)
|
| 746 |
+
res["small"] = _score_text(MODEL_SMALL, TOK_SMALL, text)
|
| 747 |
+
dt = time.perf_counter() - t0
|
| 748 |
+
|
| 749 |
+
key = {v: k for k, v in PRED_LABEL.items()}.get(colour_by, "big")
|
| 750 |
+
nlls = res[key][0]
|
| 751 |
+
|
| 752 |
+
# annotate
|
| 753 |
+
out, cursor, i = [], 0, 0
|
| 754 |
+
for m in re.finditer(r"<<(.*?)>>", text, flags=re.S):
|
| 755 |
+
out.append(html.escape(text[cursor : m.start()]))
|
| 756 |
+
if i < len(nlls):
|
| 757 |
+
p = math.exp(-nlls[i])
|
| 758 |
+
hue = 120 * min(max(p, 0.0), 1.0)
|
| 759 |
+
out.append(
|
| 760 |
+
f"<span class='cs-ch' title='p = {p*100:.1f}% · loss = {nlls[i]:.2f} nats' "
|
| 761 |
+
f"style='background:hsla({hue:.0f},70%,45%,.28)'><<{html.escape(m.group(1))}>></span>"
|
| 762 |
+
)
|
| 763 |
+
else:
|
| 764 |
+
out.append(f"<<{html.escape(m.group(1))}>>")
|
| 765 |
+
cursor = m.end()
|
| 766 |
+
i += 1
|
| 767 |
+
out.append(html.escape(text[cursor:]))
|
| 768 |
+
|
| 769 |
+
rows = []
|
| 770 |
+
for k, label, _c, note in PREDICTORS:
|
| 771 |
+
nl, hit, used = res[k]
|
| 772 |
+
if not nl:
|
| 773 |
+
continue
|
| 774 |
+
rows.append(
|
| 775 |
+
f"<tr><td>{label} <span class='cs-mnote'>{note}</span></td>"
|
| 776 |
+
f"<td>{sum(nl)/len(nl):.3f}</td>"
|
| 777 |
+
f"<td>{100*sum(hit)/len(hit):.0f}%</td><td>{len(nl)}</td></tr>"
|
| 778 |
+
)
|
| 779 |
+
truncated = res["big"][2] >= MAX_SCORE_TOKENS
|
| 780 |
+
table = (
|
| 781 |
+
CSS_BLOCK
|
| 782 |
+
+ "<div class='cs-wrap'><div class='cs-head'>Held-out choice prediction</div>"
|
| 783 |
+
"<table class='cs-tbl'><tr><th>model</th><th>loss (nats/choice)</th>"
|
| 784 |
+
f"<th>top-1</th><th>choices</th></tr>{''.join(rows)}</table>"
|
| 785 |
+
"<div class='cs-note'>Loss is the mean negative log-likelihood of the human's actual "
|
| 786 |
+
"choices — the same quantity reported in the paper (lower is more human-like). "
|
| 787 |
+
f"Scored {res['big'][2]} tokens in {dt:.1f}s"
|
| 788 |
+
+ (" (transcript truncated to fit the context budget)." if truncated else ".")
|
| 789 |
+
+ "</div></div>"
|
| 790 |
+
)
|
| 791 |
+
annotated = (
|
| 792 |
+
CSS_BLOCK
|
| 793 |
+
+ "<div class='cs-wrap'><div class='cs-head'>Green = the model expected that choice · "
|
| 794 |
+
f"red = it was surprised (colours from {html.escape(colour_by)})</div>"
|
| 795 |
+
f"<div class='cs-tx'>{''.join(out)}</div></div>"
|
| 796 |
+
)
|
| 797 |
+
return annotated, table
|
| 798 |
+
|
| 799 |
+
|
| 800 |
+
@spaces.GPU(duration=45)
|
| 801 |
+
def score_transcript(transcript: str, colour_by: str = "Qwentaur-8B") -> Tuple[str, str]:
|
| 802 |
+
"""Score every human choice in a Psych-101-formatted transcript.
|
| 803 |
+
|
| 804 |
+
Args:
|
| 805 |
+
transcript: text with human choices wrapped in `<<...>>`.
|
| 806 |
+
colour_by: which model's surprise colours the transcript.
|
| 807 |
+
Returns:
|
| 808 |
+
An annotated transcript and a per-model loss table.
|
| 809 |
+
"""
|
| 810 |
+
return _score_impl(transcript, colour_by)
|
| 811 |
+
|
| 812 |
+
|
| 813 |
+
def load_example(name: str) -> str:
|
| 814 |
+
"""Load one bundled Psych-101 human transcript into the editor."""
|
| 815 |
+
rec = EXAMPLES.get(name) or next(iter(EXAMPLES.values()))
|
| 816 |
+
return rec["text"]
|
| 817 |
+
|
| 818 |
+
|
| 819 |
+
@spaces.GPU(duration=45)
|
| 820 |
+
def run_example(name: str, colour_by: str = "Qwentaur-8B") -> Tuple[str, str, str]:
|
| 821 |
+
"""Load a bundled Psych-101 human session and score it.
|
| 822 |
+
|
| 823 |
+
Args:
|
| 824 |
+
name: which bundled human session to score.
|
| 825 |
+
colour_by: which model's surprise colours the transcript.
|
| 826 |
+
"""
|
| 827 |
+
text = load_example(name)
|
| 828 |
+
annotated, table = _score_impl(text, colour_by)
|
| 829 |
+
return text, annotated, table
|
| 830 |
+
|
| 831 |
+
|
| 832 |
+
# --------------------------------------------------------------------------------------
|
| 833 |
+
# UI
|
| 834 |
+
# --------------------------------------------------------------------------------------
|
| 835 |
+
|
| 836 |
+
CSS = """
|
| 837 |
+
#col-container { max-width: 1180px; margin: 0 auto; }
|
| 838 |
+
.dark .gradio-container { color: var(--body-text-color); }
|
| 839 |
+
"""
|
| 840 |
+
|
| 841 |
+
INTRO = """# 🧠 Centauri Cognitive Simulator
|
| 842 |
+
|
| 843 |
+
Play a real psychology experiment while three models predict, trial by trial, what a
|
| 844 |
+
**human** would do next — the fine-tuned **Qwentaur-8B**, the 13× smaller
|
| 845 |
+
**Qwentaur-0.6B**, and the **un-tuned Qwen3-8B-Base** for contrast.
|
| 846 |
+
|
| 847 |
+
The paper's headline claim is that sub-billion-parameter models already match a
|
| 848 |
+
70B Centaur at fitting human choices in-distribution — you can watch that happen here.
|
| 849 |
+
|
| 850 |
+
[Paper](https://huggingface.co/papers/2608.05224) ·
|
| 851 |
+
[Code](https://github.com/socius-org/Centauri) ·
|
| 852 |
+
[Adapters](https://huggingface.co/collections/socius/centauri-6a72e25a4669e413571fe4ac) ·
|
| 853 |
+
[Psych-101](https://huggingface.co/datasets/marcelbinz/Psych-101)
|
| 854 |
+
"""
|
| 855 |
+
|
| 856 |
+
with gr.Blocks(theme=gr.themes.Citrus(), css=CSS, title="Centauri Cognitive Simulator") as demo:
|
| 857 |
+
with gr.Column(elem_id="col-container"):
|
| 858 |
+
gr.Markdown(INTRO)
|
| 859 |
+
|
| 860 |
+
with gr.Tabs():
|
| 861 |
+
with gr.Tab("Play an experiment"):
|
| 862 |
+
state = gr.State(None)
|
| 863 |
+
with gr.Row():
|
| 864 |
+
task_radio = gr.Radio(
|
| 865 |
+
choices=[TASK_LABELS[k] for k in TASK_ORDER],
|
| 866 |
+
value=TASK_LABELS["bandit"],
|
| 867 |
+
label="Experiment",
|
| 868 |
+
scale=4,
|
| 869 |
+
)
|
| 870 |
+
start_btn = gr.Button("Start experiment", variant="primary", scale=1)
|
| 871 |
+
|
| 872 |
+
with gr.Row():
|
| 873 |
+
with gr.Column(scale=3):
|
| 874 |
+
board = gr.HTML(render_board(None))
|
| 875 |
+
with gr.Row():
|
| 876 |
+
btns = [
|
| 877 |
+
gr.Button("—", visible=False, variant="secondary") for _ in range(4)
|
| 878 |
+
]
|
| 879 |
+
chart = gr.HTML()
|
| 880 |
+
with gr.Column(scale=2):
|
| 881 |
+
preds = gr.HTML(render_predictions(None))
|
| 882 |
+
|
| 883 |
+
with gr.Accordion("What the models actually see (Psych-101 prompt)", open=False):
|
| 884 |
+
transcript = gr.Textbox(
|
| 885 |
+
label="",
|
| 886 |
+
lines=14,
|
| 887 |
+
max_lines=24,
|
| 888 |
+
show_copy_button=True,
|
| 889 |
+
interactive=False,
|
| 890 |
+
)
|
| 891 |
+
with gr.Accordion("Advanced", open=False):
|
| 892 |
+
with gr.Row():
|
| 893 |
+
seed_num = gr.Number(label="Seed", value=0, precision=0)
|
| 894 |
+
rand_seed = gr.Checkbox(label="Randomize seed", value=True)
|
| 895 |
+
blind_cb = gr.Checkbox(
|
| 896 |
+
label="Blind mode (reveal predictions only after each trial)",
|
| 897 |
+
value=False,
|
| 898 |
+
)
|
| 899 |
+
gr.Markdown(
|
| 900 |
+
"Choice probabilities are read straight off the next-token logits after "
|
| 901 |
+
"`You press <<`, renormalised over the options — no sampling, no prompt "
|
| 902 |
+
"engineering. Seeing the predictions before you act can bias you; turn on "
|
| 903 |
+
"blind mode for a cleaner test of yourself."
|
| 904 |
+
)
|
| 905 |
+
|
| 906 |
+
out_common = [
|
| 907 |
+
state,
|
| 908 |
+
board,
|
| 909 |
+
preds,
|
| 910 |
+
chart,
|
| 911 |
+
transcript,
|
| 912 |
+
seed_num,
|
| 913 |
+
*btns,
|
| 914 |
+
]
|
| 915 |
+
start_btn.click(
|
| 916 |
+
start_session,
|
| 917 |
+
inputs=[task_radio, seed_num, rand_seed, blind_cb],
|
| 918 |
+
outputs=out_common,
|
| 919 |
+
api_name="start_session",
|
| 920 |
+
)
|
| 921 |
+
for i, b in enumerate(btns):
|
| 922 |
+
b.click(
|
| 923 |
+
choose,
|
| 924 |
+
inputs=[state, gr.State(i)],
|
| 925 |
+
outputs=out_common,
|
| 926 |
+
api_name="choose" if i == 0 else False,
|
| 927 |
+
)
|
| 928 |
+
|
| 929 |
+
with gr.Tab("Score real human data"):
|
| 930 |
+
gr.Markdown(
|
| 931 |
+
"Score the models the way the paper does: how surprised is each one by the "
|
| 932 |
+
"choices a **real participant** actually made? These transcripts come "
|
| 933 |
+
"verbatim from [Psych-101]"
|
| 934 |
+
"(https://huggingface.co/datasets/marcelbinz/Psych-101) (Binz et al., "
|
| 935 |
+
"Apache-2.0), truncated to the first few trials."
|
| 936 |
+
)
|
| 937 |
+
with gr.Row():
|
| 938 |
+
preset = gr.Dropdown(
|
| 939 |
+
choices=EXAMPLE_NAMES,
|
| 940 |
+
value=EXAMPLE_NAMES[0],
|
| 941 |
+
label="Bundled human session",
|
| 942 |
+
scale=3,
|
| 943 |
+
)
|
| 944 |
+
colour_by = gr.Dropdown(
|
| 945 |
+
choices=[label for _k, label, _c, _n in PREDICTORS],
|
| 946 |
+
value="Qwentaur-8B",
|
| 947 |
+
label="Colour transcript by",
|
| 948 |
+
scale=2,
|
| 949 |
+
)
|
| 950 |
+
score_btn = gr.Button("Score choices", variant="primary", scale=1)
|
| 951 |
+
text_box = gr.Textbox(
|
| 952 |
+
label="Transcript (editable — paste your own Psych-101-style text)",
|
| 953 |
+
value=load_example(EXAMPLE_NAMES[0]),
|
| 954 |
+
lines=10,
|
| 955 |
+
max_lines=20,
|
| 956 |
+
)
|
| 957 |
+
score_table = gr.HTML()
|
| 958 |
+
annotated_out = gr.HTML()
|
| 959 |
+
|
| 960 |
+
preset.change(load_example, inputs=[preset], outputs=[text_box])
|
| 961 |
+
score_btn.click(
|
| 962 |
+
score_transcript,
|
| 963 |
+
inputs=[text_box, colour_by],
|
| 964 |
+
outputs=[annotated_out, score_table],
|
| 965 |
+
api_name="score_transcript",
|
| 966 |
+
)
|
| 967 |
+
gr.Examples(
|
| 968 |
+
examples=[[n] for n in EXAMPLE_NAMES],
|
| 969 |
+
inputs=[preset],
|
| 970 |
+
outputs=[text_box, annotated_out, score_table],
|
| 971 |
+
fn=run_example,
|
| 972 |
+
cache_examples=True,
|
| 973 |
+
cache_mode="lazy",
|
| 974 |
+
label="Real human sessions from Psych-101",
|
| 975 |
+
)
|
| 976 |
+
|
| 977 |
+
gr.Markdown(
|
| 978 |
+
"Models: [socius/Qwentaur-8B-LoRA-r16](https://huggingface.co/socius/Qwentaur-8B-LoRA-r16) "
|
| 979 |
+
"and [socius/Qwentaur-0.6B-LoRA-r16](https://huggingface.co/socius/Qwentaur-0.6B-LoRA-r16) "
|
| 980 |
+
"(LoRA adapters on Qwen3-Base, Apache-2.0) from *Small Foundation Models of Human "
|
| 981 |
+
"Cognition and Behaviour* (Oh & Gobet, 2026). Task wording and the bundled human "
|
| 982 |
+
"sessions come from Psych-101 (Binz et al., 2025)."
|
| 983 |
+
)
|
| 984 |
+
|
| 985 |
+
if __name__ == "__main__":
|
| 986 |
+
demo.launch(mcp_server=True)
|
psych101_examples.json
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"Iowa Gambling Task": {
|
| 3 |
+
"task": "steingroever2015data",
|
| 4 |
+
"experiment": "steingroever2015data/exp1.csv",
|
| 5 |
+
"participant": "2",
|
| 6 |
+
"text": "You see in front of you four decks of cards labeled U, T, L, and C.\nYou get a loan of 2000$ of play money.\nYou have to select one card at a time, from any of the four decks, for 100 trials.\nYou select a card from a deck by pressing the corresponding key.\nAfter turning a card, you win some money, the amount varies with the deck.\nYou sometimes also have to pay a penalty, which also varies with the deck.\nYour goal is to maximize profit on the loan of the play money.\n\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<L>>. You win 50.0$ and lose 50.0$.\nYou press <<U>>. You win 100.0$ and lose 300.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 1250.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<L>>. You win 50.0$ and lose 50.0$.\nYou press <<U>>. You win 100.0$ and lose 0.0$.\nYou press <<U>>. You win 100.0$ and lose 350.0$.\nYou press <<U>>. You win 100.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<U>>. You win 100.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<U>>. You win 100.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<L>>. You win 50.0$ and lose 0.0$.\nYou press <<U>>. You win 100.0$ and lose 250.0$.\nYou press <<U>>. You win 100.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<U>>. You win 100.0$ and lose 150.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<C>>. You win 50.0$ and lose 0.0$.\nYou press <<T>>. You win 100.0$ and lose 0.0$.\nYou press <<L>>. You win 50.0$ and lose 0.0$."
|
| 7 |
+
},
|
| 8 |
+
"Horizon task (2-armed bandit)": {
|
| 9 |
+
"task": "wilson2014humans",
|
| 10 |
+
"experiment": "wilson2014humans/exp1.csv",
|
| 11 |
+
"participant": "2",
|
| 12 |
+
"text": "You are participating in multiple games involving two slot machines, labeled B and E.\nThe two slot machines are different across different games.\nEach time you choose a slot machine, you get some points.\nYou choose a slot machine by pressing the corresponding key.\nEach slot machine tends to pay out about the same amount of points on average.\nYour goal is to choose the slot machines that will give you the most points across the experiment.\nThe first 4 trials in each game are instructed trials where you will be told which slot machine to choose.\nAfter these instructed trials, you will have the freedom to choose for either 1 or 6 trials.\n\nGame 1. There are 10 trials in this game.\nYou are instructed to press E and get 58 points.\nYou are instructed to press E and get 47 points.\nYou are instructed to press E and get 70 points.\nYou are instructed to press B and get 80 points.\nYou press <<B>> and get 91 points.\nYou press <<B>> and get 99 points.\nYou press <<B>> and get 83 points.\nYou press <<B>> and get 93 points.\nYou press <<B>> and get 99 points.\nYou press <<B>> and get 80 points.\n\nGame 2. There are 10 trials in this game.\nYou are instructed to press B and get 52 points.\nYou are instructed to press E and get 40 points.\nYou are instructed to press E and get 28 points.\nYou are instructed to press E and get 28 points.\nYou press <<B>> and get 40 points.\nYou press <<B>> and get 47 points.\nYou press <<B>> and get 40 points.\nYou press <<B>> and get 37 points.\nYou press <<B>> and get 30 points.\nYou press <<B>> and get 29 points.\n\nGame 3. There are 5 trials in this game.\nYou are instructed to press E and get 64 points.\nYou are instructed to press E and get 63 points.\nYou are instructed to press E and get 62 points.\nYou are instructed to press B and get 66 points.\nYou press <<E>> and get 51 points.\n\nGame 4. There are 5 trials in this game.\nYou are instructed to press E and get 63 points.\nYou are instructed to press E and get 49 points.\nYou are instructed to press B and get 44 points.\nYou are instructed to press E and get 79 points.\nYou press <<E>> and get 73 points.\n\nGame 5. There are 5 trials in this game.\nYou are instructed to press E and get 40 points.\nYou are instructed to press B and get 43 points.\nYou are instructed to press E and get 40 points.\nYou are instructed to press E and get 32 points.\nYou press <<B>> and get 50 points.\n\nGame 6. There are 10 trials in this game.\nYou are instructed to press B and get 52 points.\nYou are instructed to press E and get 53 points.\nYou are instructed to press E and get 41 points.\nYou are instructed to press B and get 69 points.\nYou press <<B>> and get 66 points.\nYou press <<B>> and get 66 points.\nYou press <<B>> and get 52 points.\nYou press <<E>> and get 25 points.\nYou press <<B>> and get 65 points.\nYou press <<B>> and get 73 points.\n\nGame 7. There are 10 trials in this game.\nYou are instructed to press B and get 25 points.\nYou are instructed to press B and get 23 points.\nYou are instructed to press E and get 62 points.\nYou are instructed to press E and get 59 points.\nYou press <<E>> and get 58 points.\nYou press <<E>> and get 68 points.\nYou press <<E>> and get 63 points.\nYou press <<E>> and get 59 points.\nYou press <<E>> and get 62 points.\nYou press <<E>> and get 53 points.\n\nGame 8. There are 10 trials in this game.\nYou are instructed to press E and get 37 points.\nYou are instructed to press E and get 43 points.\nYou are instructed to press B and get 54 points.\nYou are instructed to press E and get 49 points.\nYou press <<B>> and get 56 points.\nYou press <<B>> and get 47 points.\nYou press <<B>> and get 54 points."
|
| 13 |
+
},
|
| 14 |
+
"Intertemporal choice": {
|
| 15 |
+
"task": "ruggeri2022globalizability",
|
| 16 |
+
"experiment": "ruggeri2022globalizability/exp1.csv",
|
| 17 |
+
"participant": "2",
|
| 18 |
+
"text": "In the following you will be presented with multiple choices between two options B and C.\nPlease name which option you would prefer by pressing the corresponding key.\n\nYou have the choice between receiving 500$ immediately (press B) or receiving 550$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 500$ immediately (press B) or receiving 600$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 500$ immediately (press B) or receiving 750$ in one year (press C). You press <<B>>.\nYou have the choice between paying 500$ immediately (press B) or paying 550$ in one year (press C). You press <<B>>.\nYou have the choice between paying 500$ immediately (press B) or paying 510$ in one year (press C). You press <<B>>.\nYou have the choice between paying 500$ immediately (press B) or paying 505$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 5000$ immediately (press B) or receiving 5500$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 5000$ immediately (press B) or receiving 6000$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 5000$ immediately (press B) or receiving 7500$ in one year (press C). You press <<B>>.\nYou have the choice between receiving 500$ in one year (press B) or receiving 750$ in two years (press C). You press <<B>>.\nYou have the choice between receiving 500$ immediately (press B) or receiving 1000$ in two years (press C). You press <<B>>.\nYou have the choice between receiving 500$ immediately (press B) or receiving 750$ in one year (press C). You press <<B>>.\nYou have the choice between receive 500$ immediately (press B) or receive 750$ in one year (press C). You press <<C>>."
|
| 19 |
+
},
|
| 20 |
+
"Multi-attribute inference": {
|
| 21 |
+
"task": "hilbig2014generalized",
|
| 22 |
+
"experiment": "hilbig2014generalized/exp1.csv",
|
| 23 |
+
"participant": "3",
|
| 24 |
+
"text": "You are repeatedly presented with two options, labeled N and M.\nEach option represents a fictitious product and you have to infer which product is superior in terms of quality.\nYou select a product by pressing the corresponding key.\nFor each decision, you are provided with four expert ratings (with 1 representing a positive and 0 representing a negative rating).\nThe four experts differ in their validity.\nThe ratings of experts are given in descending order of their validity (having validities of 90%, 80%, 70%, and 60%).\n\nProduct N ratings: [0 0 1 1]. Product M ratings: [1 0 1 0]. You press <<N>>.\nProduct N ratings: [1 1 1 0]. Product M ratings: [0 0 1 0]. You press <<N>>.\nProduct N ratings: [0 1 0 0]. Product M ratings: [1 1 1 1]. You press <<M>>.\nProduct N ratings: [1 0 1 0]. Product M ratings: [0 0 1 1]. You press <<N>>.\nProduct N ratings: [0 1 1 1]. Product M ratings: [1 1 1 0]. You press <<N>>.\nProduct N ratings: [0 0 0 0]. Product M ratings: [1 1 0 0]. You press <<M>>.\nProduct N ratings: [1 1 0 1]. Product M ratings: [0 0 0 1]. You press <<N>>.\nProduct N ratings: [1 0 0 0]. Product M ratings: [0 1 1 0]. You press <<M>>.\nProduct N ratings: [0 1 0 0]. Product M ratings: [1 1 1 1]. You press <<M>>.\nProduct N ratings: [1 1 1 0]. Product M ratings: [0 1 1 1]. You press <<N>>.\nProduct N ratings: [0 1 1 0]. Product M ratings: [1 0 0 0]. You press <<N>>.\nProduct N ratings: [0 1 1 1]. Product M ratings: [1 1 1 0]. You press <<M>>.\nProduct N ratings: [1 1 1 0]. Product M ratings: [0 1 1 1]. You press <<N>>.\nProduct N ratings: [1 0 0 0]. Product M ratings: [0 0 0 1]. You press <<N>>.\nProduct N ratings: [1 1 1 0]. Product M ratings: [0 1 1 1]. You press <<N>>.\nProduct N ratings: [1 0 0 1]. Product M ratings: [0 1 1 1]. You press <<M>>.\nProduct N ratings: [0 0 0 1]. Product M ratings: [1 0 0 0]. You press <<M>>.\nProduct N ratings: [1 0 0 1]. Product M ratings: [0 1 1 1]. You press <<N>>.\nProduct N ratings: [1 1 1 1]. Product M ratings: [0 1 0 0]. You press <<N>>.\nProduct N ratings: [1 1 0 0]. Product M ratings: [0 1 0 1]. You press <<M>>.\nProduct N ratings: [0 0 0 1]. Product M ratings: [1 1 0 1]. You press <<M>>.\nProduct N ratings: [1 0 1 1]. Product M ratings: [0 0 0 0]. You press <<N>>.\nProduct N ratings: [0 1 1 0]. Product M ratings: [1 0 0 0]. You press <<N>>.\nProduct N ratings: [0 1 1 1]. Product M ratings: [1 0 0 1]. You press <<M>>.\nProduct N ratings: [1 0 0 0]. Product M ratings: [0 1 1 0]. You press <<N>>."
|
| 25 |
+
},
|
| 26 |
+
"Exploration bandit": {
|
| 27 |
+
"task": "gershman2018deconstructing",
|
| 28 |
+
"experiment": "gershman2018deconstructing/exp1.csv",
|
| 29 |
+
"participant": "2",
|
| 30 |
+
"text": "In this task, you have to repeatedly choose between two slot machines labeled O and G.\nYou can choose a slot machine by pressing its corresponding key.\nWhen you select one of the machines, you will win or lose points.\nMachine O will not always give you the same points when you select it again, but machine G will always give 0 points when you select it.\nYour goal is to choose the slot machines that will give you the most points.\nYou will receive feedback about the outcome after making a choice.\nYou will play 20 games in total, each with a different pair of slot machines.\nEach game will consist of 10 trials.\n\nGame 1:\nYou press <<O>> and get -1 points.\nYou press <<G>> and get 0 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 0 points.\nYou press <<G>> and get 0 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 2 points.\n\nGame 2:\nYou press <<O>> and get 2 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 2 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 0 points.\n\nGame 3:\nYou press <<G>> and get 0 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 0 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 2 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 2 points.\nYou press <<O>> and get 1 points.\nYou press <<O>> and get 2 points."
|
| 31 |
+
},
|
| 32 |
+
"Category learning": {
|
| 33 |
+
"task": "badham2017deficits",
|
| 34 |
+
"experiment": "badham2017deficits/exp1.csv",
|
| 35 |
+
"participant": "2",
|
| 36 |
+
"text": "You will be shown several examples of geometric objects.\nYour task is to learn a rule that allows you to tell whether an object belongs to the N or T category.\nFor each presented object, you will be asked to make a category judgment by pressing the corresponding key and then you will receive feedback.\nYou will encounter four different problems with different rules.\n\nYou encounter a new problem with a new rule determining which objects belong to each category:\nYou see a small black square. You press <<N>>. The correct category is N.\nYou see a small white square. You press <<T>>. The correct category is N.\nYou see a big white square. You press <<N>>. The correct category is N.\nYou see a big black triangle. You press <<T>>. The correct category is T.\nYou see a small white triangle. You press <<T>>. The correct category is T.\nYou see a small black triangle. You press <<T>>. The correct category is T.\nYou see a big white triangle. You press <<T>>. The correct category is T.\nYou see a big black square. You press <<N>>. The correct category is N.\nYou see a small black triangle. You press <<T>>. The correct category is T.\nYou see a small black square. You press <<N>>. The correct category is N.\nYou see a big black square. You press <<N>>. The correct category is N.\nYou see a small white triangle. You press <<T>>. The correct category is T.\nYou see a small white square. You press <<N>>. The correct category is N.\nYou see a big black triangle. You press <<T>>. The correct category is T.\nYou see a big white triangle. You press <<T>>. The correct category is T.\nYou see a big white square. You press <<N>>. The correct category is N.\nYou see a small white triangle. You press <<T>>. The correct category is T.\nYou see a big black triangle. You press <<T>>. The correct category is T.\nYou see a big white square. You press <<N>>. The correct category is N.\nYou see a small white square. You press <<N>>. The correct category is N.\nYou see a small black square. You press <<N>>. The correct category is N.\nYou see a small black triangle. You press <<T>>. The correct category is T.\nYou see a big black square. You press <<N>>. The correct category is N.\nYou see a big white triangle. You press <<T>>. The correct category is T.\nYou see a small black triangle. You press <<T>>. The correct category is T."
|
| 37 |
+
}
|
| 38 |
+
}
|
requirements.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
torch
|
| 2 |
+
transformers
|
| 3 |
+
peft
|
| 4 |
+
accelerate
|
| 5 |
+
safetensors
|