add pondernet/: PonderNet loop-thinking for MiniCPM5-2B (impl + train + eval + data + T4-verified head), separate subdir to avoid touching existing files

pondernet/README.md

@@ -0,0 +1,346 @@
+# MiniCPM5-2B PonderNet 循环思考推理改造 — 分析报告与实现
+
+> 对象仓库: `tchbcb/MiniCPM5-2B-cpu`（即 `openbmb/MiniCPM5-2B` 的副本）
+> 结论先行: **可以，而且改动比预想的小得多** —— 本目录给出可直接运行的完整实现
+> （`ponder_llama.py`，约 400 行，含训练/推理/生成全链路 + 9 组冒烟测试全部通过）。
+
+---
+
+## 一、原仓库代码分析
+
+浅克隆（`GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1`）后仓库仅 12MB，**不含任何权重**，
+也没有自定义 modeling 代码。文件清单：
+
+| 文件 | 内容 |
+|---|---|
+| `config.json` | `architectures: ["LlamaForCausalLM"]`, `model_type: "llama"` |
+| `generation_config.json` | 温度 1.0 / top_p 0.95 / eos [1, 130073] |
+| `model.safetensors.index.json` | 权重索引（381 个张量，单分片 ~5GB，未下载） |
+| `tokenizer.json` 等 | MiniCPM 词表（130,560） |
+| `README*.md` | 官方说明（transformers ≥5.6.2 原生加载，无需 trust_remote_code） |
+
+关键架构参数：**42 层 decoder、hidden 2048、16 头 / 2 KV 头（GQA）、head_dim 128、
+RoPE θ=5e6、128K 上下文、SiLU MLP（intermediate 6144）**。
+
+**分析结论**：推理逻辑 100% 位于 transformers 原生 `modeling_llama.py`，其
+`LlamaModel.forward` 是严格的"嵌入 → 42 层单遍串联 → RMSNorm → lm_head"结构，
+**没有任何递归/循环机制**。要做 PonderNet，只需在原生代码之上包一层 —— 完全不必改
+transformers 源码，也不必 fork 模型权重。
+
+## 二、PonderNet 要加什么
+
+PonderNet（Banino et al., 2021）的核心：网络对每个样本**反复执行同一计算块**，每步输出
+停机概率 λₙ，最终输出是各步输出的**概率加权混合**：
+
+```
+h_k  = Block(h_{k-1})            # 思考块第 k 步
+λ_k  = Halting(h_k)  ∈ (0,1)     # 停机概率
+w_k  = remaining · λ_k           # 本步"停下"的概率质量
+mix  = Σ w_k · h_k               # 混合输出
+remaining *= (1 - λ_k)           # 直到 Σw ≥ 1-ε 或步数上限 K
+```
+
+## 三、改造方案（本实现）
+
+### 3.1 分层：只循环"思考块"
+
+42 层切成两段：**前 34 层照常单遍**，**后 8 层（34~41）作为思考块循环执行**
+（`ponder_start_layer=34` 可调）。这样：
+
+- 简单 token 停在第 1 步 → 成本 ≈ 原模型；
+- 难 token 多想几步 → 每多想一步多 8 层的算力，粒度细、代价可控。
+
+### 3.2 四种停机信号（λ 怎么来）
+
+| 信号 | λ 定义 | 是否需训练 | 适用 |
+|---|---|---|---|
+| `entropy`（默认） | 1 − 归一化输出熵，越自信越停 | 否 | 推理即插即用 |
+| `msp` | max softmax 概率 | 否 | 同上 |
+| `drift` | 相邻两次迭代隐状态的余弦相似度超过阈值 → 已收敛 → 停 | 否 | "想不动了就停" |
+| `learned` | `nn.Linear(2048, 1)` + sigmoid，真 PonderNet 头 | **是** | 配套 ponder loss |
+
+**诚实的说明**：不训练时 `learned` 头是随机的、没有意义；推理可用的前三种是
+**启发式 halting**（借模型自身的置信度/收敛度做停机判据），效果等价于
+"自适应计算时间（ACT）+ PonderNet 式混合"。要得到论文语义的 λ，用下面提供的
+loss 只训练这个 2049 参数的头（可冻结主干）即可。
+
+### 3.3 混合与 KV cache（工程上最难的部分）
+
+每次思考迭代都会经 `DynamicCache.update()` 写 KV，直接循环会让缓存里出现重复 token。
+本实现的解法 —— **"替换式写入"**：
+
+1. 每次迭代（k≥2）先对思考块各层执行 `cache.layers[l].crop(-W)` 撤销上一步写入，
+   再跑思考块补回 —— 任意时刻缓存长度恒等于真实 token 数（已用测试逐层断言）；
+2. 停机后执行 **commit pass**：用混合表示 `h_mix` 重过一遍思考块，把"思考后"的
+   KV 写进缓存 —— **后续 token 注意到的正是思考后的表示**；
+3. 第 1 步就停机的位置自动跳过 commit（混合即单步输出，零浪费）。
+
+另一个坑：`create_causal_mask` 从非思考层读取缓存长度，对思考块会得到**多 W 的
+错误 KV 尺寸**，因此窗口 mask 必须手工构建（`_win_mask`，token 索引因果 + padding
+感知的加性浮点 mask，sdpa/eager 通用）。
+
+### 3.4 训练真正的 PonderNet（可选）
+
+`labels` 存在时自动计算 **ponder loss = CE + β·KL(w ‖ Geometric(p_g))**，
+梯度可穿过混合权重到达 λ（与论文一致）。只训练 `ponder_head`（2049 参数）即可让
+停机分布学出"难 token 多想、简单 token 少想"。
+
+### 3.5 接口与兼容性（已验证）
+
+- **权重兼容**：meta-device 对比 checkpoint 索引 —— 381 个张量键名/结构完全一致，
+  仅新增 `ponder_head.{weight,bias}` 2 个张量（2049 参数），`from_pretrained` 直接加载；
+- **`generate()` 直接可用**：继承 `LlamaForCausalLM` + 自定义 forward，HF 生成流程
+  （含左 padding、batch、`logits_to_keep`）无���任何适配；
+- **transformers 5.16.1 实测通过**（与 config 要求的 5.6.2 同代）。
+
+## 四、文件清单
+
+| 文件 | 说明 |
+|---|---|
+| `ponder_llama.py` | 核心实现：`PonderLlamaConfig` + `PonderLlamaForCausalLM`（约 400 行） |
+| `test_ponder.py` | 随机权重冒烟测试，9 组用例（cache 一致性/生成/训练反传/4 信号/eager/左 padding） |
+| `test_train.py` | 训练管线验证，5 组用例（数据/三种模式/保存重载/KL 字段） |
+| `test_gpu_readiness.py` | **T4/fp16 就绪性预验证**，5 组用例（fp16 dtype 链/KL 不 NaN/ckpt 一致性） |
+| `train_ponder_head.py` | **halting 头微调脚本**（head / head+lora / head+block 三种模式，含难度分级数据生成器） |
+| `eval_ponder.py` | **效果评估**：难度分桶 CE/步数统计、训前训后对比（`--compare`）、思考收益（`--k1-baseline`） |
+| `ponder_steps_demo.py` | tiny 模型演示：训练中平均思考步数向几何先验漂移的轨迹 |
+| `demo_minicpm5.py` | 真实权重 demo：chat 对比、每 token 思考步数统计 |
+| `run_t4.sh` | **T4 一键脚本**：自检→下载权重→数据→训练→评估全自动串联 |
+| `data/pondernet_train.jsonl` | 难度分级训练集 600 条（easy 220 / medium 200 / hard 180，答案反向构造保证正确） |
+| `data/pondernet_train.eval.jsonl` | 难度分级评估集 85 条（与训练集无重叠） |
+
+## 五、快速上手
+
+```python
+from ponder_llama import PonderLlamaForCausalLM, PonderLlamaConfig, LlamaConfig
+
+cfg = PonderLlamaConfig.from_llama(
+    LlamaConfig.from_pretrained("/path/to/MiniCPM5-2B-cpu"),
+    ponder_signal="entropy",     # 或 msp / drift / learned
+    max_ponder_steps=8,
+    ponder_start_layer=None,     # None = 最后 8 层作思考块
+)
+model = PonderLlamaForCausalLM.from_pretrained(
+    "/path/to/MiniCPM5-2B-cpu", config=cfg,
+    torch_dtype="bfloat16", low_cpu_mem_usage=True).eval().cuda()
+
+out = model(input_ids, output_ponder=True)
+print(out.ponder_steps)     # 每个位置实际思考了几步
+print(model._ponder_log)    # 每次 forward 的诊断日志
+```
+
+训练 halting 头（可选，其余全部冻结）：
+
+```python
+for n, p in model.named_parameters():
+    p.requires_grad = n.startswith("ponder_head")
+cfg.ponder_signal = "learned"        # 切到可训练信号
+# 正常 forward(input_ids, labels=...) → loss 里已含 KL(‖Geometric) 正则
+```
+
+主要超参：
+
+| 参数 | 默认 | 含义 |
+|---|---|---|
+| `ponder_start_layer` | L−8 | 思考块起始层 |
+| `max_ponder_steps` | 8 | 思考步数上限 K |
+| `ponder_epsilon` | 0.01 | 累计质量 1−ε 即停 |
+| `ponder_signal` | entropy | 停机信号 |
+| `ponder_window` | 1 | prefill 思考窗口（=1 即"回答前思考"） |
+| `ponder_all_positions` | False | True = 所有位置独立 halting |
+| `commit_kv` | True | 用混合表示写 KV（False 省一遍计算，语义略糙） |
+| `ponder_prior_p` / `ponder_loss_beta` | 0.5 / 0.01 | 几何先验 p_g 与 KL 权重 β |
+
+## 六、训练 halting 头（真 PonderNet）
+
+### 6.1 三种模式与代价
+
+| 模式 | 可训练参数 | 显存 | 语义 |
+|---|---|---|---|
+| `head` | 2049 | 极低（8GB 单卡可跑） | 仅学"何时停"，思考块本身不变 |
+| `head+lora`（推荐） | 头 + LoRA（r=16，仅思考块） | 低 | 思考块学会"被重复执行"，完整 PonderNet |
+| `head+block` | 头 + 思考块全参 | 高 | 效果上限最高 |
+
+```bash
+# 生成演示数据（16 条: 简单问答 + 推理题, 让头有"难题多想"的信号可学）
+python train_ponder_head.py --model /path/to/MiniCPM5-2B-cpu --make-demo-data demo.jsonl
+
+# 推荐: 头 + 思考块 LoRA
+python train_ponder_head.py --model /path/to/MiniCPM5-2B-cpu \
+    --train-mode head+lora --lora-scope ponder --lora-r 16 \
+    --data demo.jsonl --epochs 2 --batch-size 2 --accum 4 \
+    --prior-p 0.4 --beta 0.05 --head-bias-init -1.0 \
+    --output out/ponder-lora --log-every 5
+```
+
+数据格式（jsonl，只对 assistant 回答区间计 loss）：
+```json
+{"messages": [{"role": "user", "content": "9.11 和 9.9 哪个大？"},
+              {"role": "assistant", "content": "9.9 更大。因为 0.90 > 0.11 …"}]}
+```
+
+### 6.2 关键超参与预期现象
+
+- **`--prior-p`（几何先验 p_g）**：控制"想几步"的先验预期，期望步数 ≈ 1/p_g。
+  想让模型平均思考 3 步左右 → p_g=0.3；2.5 步 → p_g=0.4。
+- **`--head-bias-init`**：初始停机偏置，sigmoid(-1.0)≈0.27（初始期望约 3.7 步）。
+- **`--beta`**：KL 权重。太小 → 头被 CE 拉着全跑满 K 步；太大 → 全挤在第 1 步。
+
+训练日志中三列数字的联动（tiny 模型实测轨迹）：
+
+```
+  step |      CE |      KL | 平均思考步数
+     0 |  5.4395 |  0.0390 |       1.99
+    60 |  4.2772 |  0.0083 |       2.31   ← p_g=0.4, 期望 2.5
+```
+
+CE 下降（任务能力）+ KL 下降（停机分布靠向几何先验）+ 步数向 1/p_g 漂移但保留
+区分度 —— 三者同时发生才是健康的 PonderNet 训练；如果步数立刻塌到 1 或钉死在 K，
+调 p_g / beta / head_bias_init。
+
+### 6.3 训练产物与推理加载
+
+```python
+# head / head+lora 模式产物: ponder_head.safetensors (+ adapter_model.safetensors)
+from safetensors.torch import load_file
+model = PonderLlamaForCausalLM.from_ponder(
+    "/path/to/MiniCPM5-2B-cpu",
+    ponder_kwargs={"ponder_signal": "learned", "max_ponder_steps": 8})
+head = load_file("out/ponder-lora/ponder_head.safetensors")
+model.ponder_head.load_state_dict({k.replace("ponder_head.", ""): v
+                                   for k, v in head.items()})
+# LoRA 适配器: model = PeftModel.from_pretrained(model, "out/ponder-lora")
+```
+
+## 七、T4 实战 runbook（16GB 显存）
+
+### 7.1 为什么 T4 需要专门适配
+
+T4 是 pre-Ampere 卡（sm_75）：**不支持 bf16，只能 fp16 + GradScaler**。本仓库已针对
+真实半精度 GPU 预修复并预验证（`test_gpu_readiness.py`，CPU fp16 复现）：
+
+1. fp16/bf16 模型 + fp32 ponder_head 的 dtype 转换链（修复前直接报 dtype 不匹配）。
+2. fp16 下 KL 正则的 NaN 风险：停机质量 w 中的小值在 fp16 域取 log 会下溢出 NaN，
+   KL 现已固定在 fp32 域计算。
+3. 全部停机信号的 λ 统一 cast 回隐状态 dtype，混合链路 dtype 一致。
+4. 思考块 gradient checkpointing：思考循环最多 K 次前向，激活是普通前向的 K 倍，
+   重算是 16GB 显存训练的关键（开启后开/关 loss 与梯度完全一致）。
+
+显存预算（head+lora, fp16, batch 2×384）：权重 5.0GB + LoRA/头 ≈ 0.1GB +
+激活（重算后）≈ 1GB + logits（词表 130560）≈ 1.5GB ≈ **8GB 左右，T4 富余**。
+
+### 7.2 一键运行
+
+```bash
+bash run_t4.sh          # 自检 → 下载权重(约5GB, 磁盘不足自动清理缓存) →
+                        # 数据 → 训练(fp16) → 训前/训后评估 + 思考收益
+```
+
+### 7.3 训后看什么（评估表怎么读）
+
+```bash
+python eval_ponder.py --model $MODEL_DIR \
+    --head out/ponder-t4/ponder_head.safetensors --adapter out/ponder-t4 \
+    --data data/pondernet_train.eval.jsonl --compare --k1-baseline
+```
+
+- **难度分化的 avg_steps**：健康的训练应看到 `easy < medium < hard` 的思考步数梯度
+  （数据集本身就是按难度分桶构造的）；训前（bias=-1, λ≈0.27）各桶步数几乎相同。
+- **ΔCE（思考收益）**：`--k1-baseline` 输出 CE(K=1) − CE(K=8)，正值说明"多想"确实
+  降低了损失；重点看 hard 桶是否比 easy 桶收益更大（思考用在刀刃上）。
+- **训练日志**：CE 下降 + KL 下降 + 步数向 1/p_g≈2.5 漂移但保留区分度。
+
+## 八、局限与提醒
+
+1. **免训练信号是启发式**：`entropy/msp/drift` 停机判据并非模型学出来的元认知，
+   实测大概率每个位置都会跑满 K 步。要得到"该多则多、该少则少"的停机行为，
+   用第六节的脚本微调 halting 头（数据量要求很小，head 模式单卡即可），
+   或在思考块上加 LoRA 联合训练 —— 那才是完整意义的 PonderNet。
+2. **算力换深度**：每多想一步 = 多 8 层前向（约 19% 的整模型算力）。decode 阶段
+   逐 token 思考会显著降速，可把 `max_ponder_steps` 调小或只保留 prefill 思考。
+3. **语义偏移风险**：思考块是"重复过同样的层"，本质是给特定 token 动态加深网络。
+   原模型没在这种模式下训练过，输出分布可能有轻微偏移（类似 Universal Transformer
+   的权重共享循环）。混合权重 `w` 与 commit-KV 机制保证了数学上的自洽，但不保证
+   生成质量提升 —— 建议以 `max_ponder_steps=1`（≈原模型）为 baseline 对比评估。
+4. 本实现针对 transformers 5.x（5.16.1 实测）；4.x 的 Cache API 不同，需要小幅适配。
+
+## 九、T4 实测结果（2026-09-08，Colab Tesla T4 16GB）
+
+环境：Python 3.13 / torch 2.11.0+cu128 / transformers 5.16.1 / peft 0.20.0。
+600 条难度分级训练数据（easy 220 / medium 200 / hard 180），85 条评估 + 50 条 OOD 压力题。
+
+### 9.1 训练（head+lora, LoRA r=16 仅注入最后 8 层思考块, 2 epochs, fp16+GradScaler）
+
+- 总耗时 **617 s**（含加载与保存），无 NaN、无 OOM（batch 2×512 + grad-ckpt）
+- CE：2.50 → 0.03~0.05（epoch 2 稳定在 0.01–0.05 量级）
+- 平均思考步数（训练期）：2.29 → ~1.30，说明 halting 头确实在 CE+KL 联合作用下
+  从"多想"向"够用即停"移动
+- 产物：`out/ponder_lora/`（adapter_model.safetensors 19MB + ponder_head.safetensors 8.4KB）
+
+### 9.2 评估集结果（85 条）
+
+| 配置 | 全局 CE | easy 步数 | medium 步数 | hard 步数 |
+|---|---|---|---|---|
+| 训前（bias=-1，λ≈0.27） | 8.821 | 7.23 | 7.20 | 7.34 |
+| 训后 K=8 自适应 | **0.018** | 1.46 | 1.30 | 1.20 |
+| 训后 K=1（强制单步） | 0.030 | 1.00 | 1.00 | 1.00 |
+
+三个核心观察：
+
+1. **自适应计算带来真实质量增益**：K=8 自适应（���均仅 1.3 步）CE 0.018，比强制
+   单步 baseline 的 0.030 低 38%。即模型用约 1/8 的"跑满算力"成本取得了比单步
+   更低的损失 —— 这正是 PonderNet 论文声称的"质量-算力帕累托改进"在真实 2B 模型上的复现。
+2. **训前 vs 训后对照**：训前 head 是"无脑跑满 K=8"（7.2–7.3 步，无难度区分）；
+   训后绝大多数位置 1 步即停（77%–91%），且保留了少量多步位置 —— 停机行为完全被训练重塑。
+3. **步数分化的方向**：easy > medium > hard（1.46 > 1.30 > 1.20）。这与"难题多想"
+   的直觉相反，但与 PonderNet 的实际优化目标一致 —— 步数跟随的是 **per-token 预测
+   不确定性**：hard 题答案高度模板化（"设鸡 x 只…"），每个 token 都好预测，1 步就够；
+   easy 里的翻译题英文词更难预测，反而多想一点。
+
+### 9.3 OOD 压力测试（50 条训外难题）
+
+| 题型 | 平均步数 | 停在 1 步占比 |
+|---|---|---|
+| 数字串复述（10–14 位，纯记忆压力） | **1.50** | 66% |
+| 平方（12–99） | 1.34 | 81% |
+| 三步混合运算 a×b+c×d | 1.28 | 87% |
+| 逻辑链比较 | 1.26 | 89% |
+| 大数乘法（3–4 位） | 1.18 | 90% |
+| 三步应用题（折扣×数量） | 1.17 | 89% |
+
+- OOD 上 K=8 CE 0.705 vs K=1 CE 0.724：思考仍有增益（+2.7%），且步数最高的恰是
+  **记忆压力最大的数字串复述** —— halting 头学到的信号与"表示不确定性"正相关，
+  方向正确。
+- 但步数整体仍低（≤1.5）：600 条简单数据上 CE 的最优解就是"快停"，模型没有遇到
+  "1 步真解不出"的压力。
+
+### 9.4 结论与下一步
+
+**已验证**：PonderNet 循环思考在真实 2B 模型 + T4 上端到端可行 —— 训练稳定、
+自适应停机、质量优于单步 baseline、算力成本可控（~10 分钟可训完）。
+
+**未出现**：按题目语义难度分桶的步数分化。要让"难题多想"真正显现，需要：
+
+1. **1 步真实解不出的数据**：例如把 CoT 中间推理步纳入监督（而非只给模板化最终答案）、
+   或训练集包含模型当前能力边界之外的任务；
+2. **更大的 prior 压力**：`--prior-p` 调小（如 0.2）或加大 β，强制步数分布远离 1；
+3. **两阶段训练**：先 LoRA 拟合任务，再冻结 LoRA 单独训 head，避免"快停"在
+   训练早期就成为 CE 最优解。
+
+### 9.5 T4 复现命令
+
+```bash
+# T4 端（Colab）—— 数据 md5: mix.jsonl=41c7ede4, mix.eval.jsonl=5a03ce37（与本地一致）
+pip uninstall -y torchao            # peft 0.20 与 Colab 预装 torchao 0.10 冲突
+python train_ponder_head.py --model models/MiniCPM5-2B \
+    --train-mode head+lora --lora-scope ponder --data data/mix.jsonl \
+    --dtype float16 --epochs 2 --batch-size 2 --accum 4 --grad-ckpt \
+    --output out/ponder_lora
+python eval_ponder.py --model models/MiniCPM5-2B --data data/mix.eval.jsonl \
+    --head out/ponder_lora/ponder_head.safetensors --adapter out/ponder_lora \
+    --max-steps 8 --generate
+```
+
+注意：新版 torch 的 `is_bf16_supported()` 对 T4 (sm75) 返回 True 但那是模拟 bf16
+（无硬件 tensor core），训练脚本已改为按 compute capability 判断，T4 自动落
+fp16 + GradScaler。

pondernet/data/mix.eval.jsonl

@@ -0,0 +1,85 @@
+{"messages": [{"role": "user", "content": "把'山'翻译成英文"}, {"role": "assistant", "content": "山 的英文是 mountain。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "6 - 6 等于几？"}, {"role": "assistant", "content": "6 - 6 = 0。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "4 - 1 等于几？"}, {"role": "assistant", "content": "4 - 1 = 3。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'doctor'翻译成中文"}, {"role": "assistant", "content": "doctor 的中文是 医生。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'水'翻译成英文"}, {"role": "assistant", "content": "水 的英文是 water。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'computer'翻译成中文"}, {"role": "assistant", "content": "computer 的中文是 电脑。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'egg'翻译成中文"}, {"role": "assistant", "content": "egg 的中文是 鸡蛋。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "1 + 3 等于几？"}, {"role": "assistant", "content": "1 + 3 = 4。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "4 + 6 等于几？"}, {"role": "assistant", "content": "4 + 6 = 10。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "5 - 5 等于几？"}, {"role": "assistant", "content": "5 - 5 = 0。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "7 + 2 等于几？"}, {"role": "assistant", "content": "7 + 2 = 9。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'花'翻译成英文"}, {"role": "assistant", "content": "花 的英文是 flower。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'evening'翻译成中文"}, {"role": "assistant", "content": "evening 的中文是 晚上。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'blue'翻译成中文"}, {"role": "assistant", "content": "blue 的中文是 蓝色。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "3 + 9 等于几？"}, {"role": "assistant", "content": "3 + 9 = 12。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "把'flower'翻译成中文"}, {"role": "assistant", "content": "flower 的中文是 花。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "6 + 8 等于几？"}, {"role": "assistant", "content": "6 + 8 = 14。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "8 + 4 等于几？"}, {"role": "assistant", "content": "8 + 4 = 12。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "3 + 1 等于几？"}, {"role": "assistant", "content": "3 + 1 = 4。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "6 + 2 等于几？"}, {"role": "assistant", "content": "6 + 2 = 8。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "2 + 7 等于几？"}, {"role": "assistant", "content": "2 + 7 = 9。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "9 + 7 等于几？"}, {"role": "assistant", "content": "9 + 7 = 16。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "3 - 3 等于几？"}, {"role": "assistant", "content": "3 - 3 = 0。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "7 + 5 等于几？"}, {"role": "assistant", "content": "7 + 5 = 12。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "6 + 5 等于几？"}, {"role": "assistant", "content": "6 + 5 = 11。"}], "difficulty": "easy"}
+{"messages": [{"role": "user", "content": "4560 分钟 等于多少 小时？"}, {"role": "assistant", "content": "4560 分钟 = 76 小时。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "710 角 等于多少 元？"}, {"role": "assistant", "content": "710 角 = 71 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 11 元，买 11 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 11 元，数量 11 支，一共 11 × 11 = 121 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "387 加 965 等于多少？"}, {"role": "assistant", "content": "387 + 965 = 1352。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "1560 分钟 等于多少 小时？"}, {"role": "assistant", "content": "1560 分钟 = 26 小时。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "16 千克 等于多少 克？"}, {"role": "assistant", "content": "16 千克 = 16000 克。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "65 米 等于多少 厘米？"}, {"role": "assistant", "content": "65 米 = 6500 厘米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一辆汽车每小时行驶 40 千米，行驶 3 小时，一共行驶多少千米？"}, {"role": "assistant", "content": "速度 40 千米/小时 × 时间 3 小时 = 120 千米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "56 千克 等于多少 克？"}, {"role": "assistant", "content": "56 千克 = 56000 克。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "80 米 等于多少 厘米？"}, {"role": "assistant", "content": "80 米 = 8000 厘米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 11 元，买 2 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 11 元，数量 2 支，一共 11 × 2 = 22 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "30000 毫米 等于多少 米？"}, {"role": "assistant", "content": "30000 毫米 = 30 米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 17 元，买 9 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 17 元，数量 9 支，一共 17 × 9 = 153 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 2 元，买 8 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 2 元，数量 8 支，一共 2 × 8 = 16 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 10 元，买 2 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 10 元，数量 2 支，一共 10 × 2 = 20 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "970 角 等于多少 元？"}, {"role": "assistant", "content": "970 角 = 97 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 14 元，买 8 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 14 元，数量 8 支，一共 14 × 8 = 112 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 20 元，买 10 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 20 元，数量 10 支，一共 20 × 10 = 200 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "57 元 等于多少 角？"}, {"role": "assistant", "content": "57 元 = 570 角。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "29 乘 3 等于多少？"}, {"role": "assistant", "content": "29 × 3 = 87。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "53 米 等于多少 厘米？"}, {"role": "assistant", "content": "53 米 = 5300 厘米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 6 元，买 9 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 6 元，数量 9 支，一共 6 × 9 = 54 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "98 乘 7 等于多少？"}, {"role": "assistant", "content": "98 × 7 = 686。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "38 元 等于多少 角？"}, {"role": "assistant", "content": "38 元 = 380 角。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "58000 毫米 等于多少 米？"}, {"role": "assistant", "content": "58000 毫米 = 58 米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "83 元 等于多少 角？"}, {"role": "assistant", "content": "83 元 = 830 角。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "87 乘 4 等于多少？"}, {"role": "assistant", "content": "87 × 4 = 348。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一辆汽车每小时行驶 70 千米，行驶 8 小时，一共行驶多少千米？"}, {"role": "assistant", "content": "速度 70 千米/小时 × 时间 8 小时 = 560 千米。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "293 减 281 等于多少？"}, {"role": "assistant", "content": "293 - 281 = 12。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "一支笔 10 元，买 7 支一共需要多少钱？"}, {"role": "assistant", "content": "单价 10 元，数量 7 支，一共 10 × 7 = 70 元。"}], "difficulty": "medium"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 35 个头，从下面数有 100 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 35 - x 只。列方程 2x + 4(35 - x) = 100，化简得 140 - 2x = 100，x = 20。所以鸡 20 只，兔 15 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "甲、乙两车分别从相距 480 千米的两地同时出发，相向而行。甲车每小时行 30 千米，乙车每小时行 50 千米，几小时后两车相遇？"}, {"role": "assistant", "content": "两车速度和为 30 + 50 = 80 千米/小时。相遇时间 = 距离 ÷ 速度和 = 480 ÷ 80 = 6 小时。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "356 乘 47 等于多少？"}, {"role": "assistant", "content": "356 × 47 = 16732。把 47 拆成 40 + 7：356 × 40 = 14240，356 × 7 = 2492，相加得 16732 = 16732。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：10, 17, 24, 31, 38，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差恒为 7，下一个数是 38 + 7 = 45。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 25 个头，从下面数有 88 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 25 - x 只。列方程 2x + 4(25 - x) = 88，化简得 100 - 2x = 88，x = 6。所以鸡 6 只，兔 19 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 22 个头，从下面数有 70 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 22 - x 只。列方程 2x + 4(22 - x) = 70，化简得 88 - 2x = 70，x = 9。所以鸡 9 只，兔 13 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "178 乘 30 等于多少？"}, {"role": "assistant", "content": "178 × 30 = 5340。把 30 拆成 30 + 0：178 × 30 = 5340，178 × 0 = 0，相加得 5340 = 5340。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "604 乘 34 等于多少？"}, {"role": "assistant", "content": "604 × 34 = 20536。把 34 拆成 30 + 4：604 × 30 = 18120，604 × 4 = 2416，相加得 20536 = 20536。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 26 个头，从下面数有 72 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 26 - x 只。列方程 2x + 4(26 - x) = 72，化简得 104 - 2x = 72，x = 16。所以鸡 16 只，兔 10 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "348 乘 44 等于多少？"}, {"role": "assistant", "content": "348 × 44 = 15312。把 44 拆成 40 + 4：348 × 40 = 13920，348 × 4 = 1392，相加得 15312 = 15312。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 15 个头，从下面数有 52 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 15 - x 只。列方程 2x + 4(15 - x) = 52，化简得 60 - 2x = 52，x = 4。所以鸡 4 只，兔 11 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "甲、乙两车分别从相距 280 千米的两地同时出发，相向而行。甲车每小时行 20 千米，乙车每小时行 50 千米，几小时后两车相遇？"}, {"role": "assistant", "content": "两车速度和为 20 + 50 = 70 千米/小时。相遇时间 = 距离 ÷ 速度和 = 280 ÷ 70 = 4 小时。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 18 个头，从下面数有 54 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 18 - x 只。列方程 2x + 4(18 - x) = 54，化简得 72 - 2x = 54，x = 9。所以鸡 9 只，兔 9 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：16, 21, 26, 31, 36，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差恒为 5，下一个数是 36 + 5 = 41。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "500 克糖水中含糖 100 克，这种糖水的浓度是多少？"}, {"role": "assistant", "content": "浓度 = 糖 ÷ 糖水 = 100 ÷ 500 = 20%。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：8, 11, 15, 20, 26，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差依次是 3, 4, 5, 6，差递增 1。下一个差是 8，下一个数是 26 + 8 = 34。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：17, 25, 33, 41, 49，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差恒为 8，下一个数是 49 + 8 = 57。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "426 除以 10，商和余数分别是多少？"}, {"role": "assistant", "content": "426 ÷ 10 = 42 余 6，即商是 42，余数是 6。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "336 乘 61 等于多少？"}, {"role": "assistant", "content": "336 × 61 = 20496。把 61 拆成 60 + 1：336 × 60 = 20160，336 × 1 = 336，相加得 20496 = 20496。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "500 克糖水中含糖 125 克，这种糖水的浓度是多少？"}, {"role": "assistant", "content": "浓度 = 糖 ÷ 糖水 = 125 ÷ 500 = 25%。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：14, 22, 30, 38, 46，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差恒为 8，下一个数是 46 + 8 = 54。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 10 个头，从下面数有 28 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 10 - x 只。列方程 2x + 4(10 - x) = 28，化简得 40 - 2x = 28，x = 6。所以鸡 6 只，兔 4 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "250 除以 6，商和余数分别是多少？"}, {"role": "assistant", "content": "250 ÷ 6 = 41 余 4，即商是 41，余数是 4。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "甲、乙两车分别从相距 350 千米的两地同时出发，相向而行。甲车每小时行 40 千米，乙车每小时行 30 千米，几小时后两车相遇？"}, {"role": "assistant", "content": "两车速度和为 40 + 30 = 70 千米/小时。相遇时间 = 距离 ÷ 速度和 = 350 ÷ 70 = 5 小时。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "128 乘 98 等于多少？"}, {"role": "assistant", "content": "128 × 98 = 12544。把 98 拆成 90 + 8：128 × 90 = 11520，128 × 8 = 1024，相加得 12544 = 12544。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：9, 12, 17, 24, 33，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差依次是 3, 5, 7, 9，差递增 2。下一个差是 13，下一个数是 33 + 13 = 46。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "甲、乙两车分别从相距 150 千米的两地同时出发，相向而行。甲车每小时行 20 千米，乙车每小时行 30 千米，几小时后两车相遇？"}, {"role": "assistant", "content": "两车速度和为 20 + 30 = 50 千米/小时。相遇时间 = 距离 ÷ 速度和 = 150 ÷ 50 = 3 小时。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "找规律：15, 27, 39, 51, 63，下一个数是多少？"}, {"role": "assistant", "content": "相邻两项的差恒为 12，下一个数是 63 + 12 = 75。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "鸡兔同笼，从上面数有 8 个头，从下面数有 26 只脚，鸡和兔各有几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 8 - x 只。列方程 2x + 4(8 - x) = 26，化简得 32 - 2x = 26，x = 3。所以鸡 3 只，兔 5 只。"}], "difficulty": "hard"}
+{"messages": [{"role": "user", "content": "404 乘 87 等于多少？"}, {"role": "assistant", "content": "404 × 87 = 35148。把 87 拆成 80 + 7：404 × 80 = 32320，404 × 7 = 2828，相加得 35148 = 35148。"}], "difficulty": "hard"}

pondernet/data/ood_hard.jsonl

@@ -0,0 +1,50 @@
+{"messages": [{"role": "user", "content": "17 的平方是多少？"}, {"role": "assistant", "content": "17² = 17 × 17 = 289。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "戊 比 丁 高，丁 比 乙 低，乙 比 甲 高。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：戊=39，丁=29，乙=82，甲=31，所以 乙 最高。"}], "difficulty": "ood_logicchain"}
+{"messages": [{"role": "user", "content": "9973 乘 3302 等于多少？"}, {"role": "assistant", "content": "9973 × 3302 = 32930846。把 3302 拆成 3300 + 2：9973 × 3300 = 32910900，9973 × 2 = 19946，相加得 32930846 = 32930846。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "4088 乘 3222 等于多少？"}, {"role": "assistant", "content": "4088 × 3222 = 13171536。把 3222 拆成 3200 + 22：4088 × 3200 = 13081600，4088 × 22 = 89936，相加得 13171536 = 13171536。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "53 × 57 + 37 × 46 等于多少？"}, {"role": "assistant", "content": "53 × 57 = 3021，37 × 46 = 1702，相加得 3021 + 1702 = 4723。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "一件商品原价 85 元，打 0 折出售，小明买了 12 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 85 × 0.9 = 76 元，共 76 × 12 = 912 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "91 × 27 + 33 × 79 等于多少？"}, {"role": "assistant", "content": "91 × 27 = 2457，33 × 79 = 2607，相加得 2457 + 2607 = 5064。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：3545315855"}, {"role": "assistant", "content": "3545315855。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：013234407587"}, {"role": "assistant", "content": "013234407587。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "28 × 22 + 61 × 36 等于多少？"}, {"role": "assistant", "content": "28 × 22 = 616，61 × 36 = 2196，相加得 616 + 2196 = 2812。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：9942782302"}, {"role": "assistant", "content": "9942782302。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "41 × 64 + 25 × 49 等于多少？"}, {"role": "assistant", "content": "41 × 64 = 2624，25 × 49 = 1225，相加得 2624 + 1225 = 3849。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：975998899632"}, {"role": "assistant", "content": "975998899632。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "3208 乘 7748 等于多少？"}, {"role": "assistant", "content": "3208 × 7748 = 24855584。把 7748 拆成 7700 + 48：3208 × 7700 = 24701600，3208 × 48 = 153984，相加得 24855584 = 24855584。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "28 的平方是多少？"}, {"role": "assistant", "content": "28² = 28 × 28 = 784。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：4684199908"}, {"role": "assistant", "content": "4684199908。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "一件商品原价 175 元，打 0 折出售，小明买了 13 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 175 × 0.9 = 157 元，共 157 × 13 = 2041 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "5849 乘 4197 等于多少？"}, {"role": "assistant", "content": "5849 × 4197 = 24548253。把 4197 拆成 4100 + 97：5849 × 4100 = 23980900，5849 × 97 = 567353，相加得 24548253 = 24548253。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "47 的平方是多少？"}, {"role": "assistant", "content": "47² = 47 × 47 = 2209。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "一件商品原价 31 元，打 0 折出售，小明买了 9 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 31 × 0.9 = 27 元，共 27 × 9 = 243 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "34 × 22 + 99 × 69 等于多少？"}, {"role": "assistant", "content": "34 × 22 = 748，99 × 69 = 6831，相加得 748 + 6831 = 7579。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "8368 乘 3565 等于多少？"}, {"role": "assistant", "content": "8368 × 3565 = 29831920。把 3565 拆成 3500 + 65：8368 × 3500 = 29288000，8368 × 65 = 543920，相加得 29831920 = 29831920。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "95 × 59 + 66 × 34 等于多少？"}, {"role": "assistant", "content": "95 × 59 = 5605，66 × 34 = 2244，相加得 5605 + 2244 = 7849。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "53 的平方是多少？"}, {"role": "assistant", "content": "53² = 53 × 53 = 2809。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "6845 乘 7205 等于多少？"}, {"role": "assistant", "content": "6845 × 7205 = 49318225。把 7205 拆成 7200 + 5：6845 × 7200 = 49284000，6845 × 5 = 34225，相加得 49318225 = 49318225。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "一件商品原价 56 元，打 0 折出售，小明买了 12 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 56 × 0.9 = 50 元，共 50 × 12 = 600 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "97 × 19 + 71 × 24 等于多少？"}, {"role": "assistant", "content": "97 × 19 = 1843，71 × 24 = 1704，相加得 1843 + 1704 = 3547。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "15 × 56 + 41 × 17 等于多少？"}, {"role": "assistant", "content": "15 × 56 = 840，41 × 17 = 697，相加得 840 + 697 = 1537。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "73 × 34 + 83 × 82 等于多少？"}, {"role": "assistant", "content": "73 × 34 = 2482，83 × 82 = 6806，相加得 2482 + 6806 = 9288。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "乙 比 戊 高，戊 比 甲 高，甲 比 丁 高。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：乙=92，戊=62，甲=48，丁=34，所以 乙 最高。"}], "difficulty": "ood_logicchain"}
+{"messages": [{"role": "user", "content": "63 的平方是多少？"}, {"role": "assistant", "content": "63² = 63 × 63 = 3969。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "294 乘 3138 等于多少？"}, {"role": "assistant", "content": "294 × 3138 = 922572。把 3138 拆成 3100 + 38：294 × 3100 = 911400，294 × 38 = 11172，相加得 922572 = 922572。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "38 × 83 + 55 × 86 等于多少？"}, {"role": "assistant", "content": "38 × 83 = 3154，55 × 86 = 4730，相加得 3154 + 4730 = 7884。"}], "difficulty": "ood_multistep"}
+{"messages": [{"role": "user", "content": "3752 乘 6781 等于多少？"}, {"role": "assistant", "content": "3752 × 6781 = 25442312。把 6781 拆成 6700 + 81：3752 × 6700 = 25138400，3752 × 81 = 303912，相加得 25442312 = 25442312。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "戊 比 丁 高，丁 比 乙 高，乙 比 丙 低。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：戊=85，丁=39，乙=38，丙=92，所以 丙 最高。"}], "difficulty": "ood_logicchain"}
+{"messages": [{"role": "user", "content": "一件商品原价 170 元，打 0 折出售，小明买了 3 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 170 × 0.9 = 153 元，共 153 × 3 = 459 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "5871 乘 2163 等于多少？"}, {"role": "assistant", "content": "5871 × 2163 = 12698973。把 2163 拆成 2100 + 63：5871 × 2100 = 12329100，5871 × 63 = 369873，相加得 12698973 = 12698973。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "一件商品原价 24 元，打 -10 折出售，小明买了 8 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 24 × 0.8 = 19 元，共 19 × 8 = 152 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "一件商品原价 184 元，打 -10 折出售，小明买了 5 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 184 × 0.8 = 147 元，共 147 × 5 = 735 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：062073159646"}, {"role": "assistant", "content": "062073159646。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：51800279567"}, {"role": "assistant", "content": "51800279567。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "51 的平方是多少？"}, {"role": "assistant", "content": "51² = 51 × 51 = 2601。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "请原样重复这串数字：4610614728401"}, {"role": "assistant", "content": "4610614728401。"}], "difficulty": "ood_digitcopy"}
+{"messages": [{"role": "user", "content": "8033 乘 1931 等于多少？"}, {"role": "assistant", "content": "8033 × 1931 = 15511723。把 1931 拆成 1900 + 31：8033 × 1900 = 15262700，8033 × 31 = 249023，相加得 15511723 = 15511723。"}], "difficulty": "ood_bigmul"}
+{"messages": [{"role": "user", "content": "丁 比 甲 低，甲 比 丙 低，丙 比 乙 低。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：丁=70，甲=72���丙=79，乙=82，所以 乙 最高。"}], "difficulty": "ood_logicchain"}
+{"messages": [{"role": "user", "content": "26 的平方是多少？"}, {"role": "assistant", "content": "26² = 26 × 26 = 676。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "甲 比 戊 高，戊 比 丙 高，丙 比 乙 低。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：甲=91，戊=25，丙=14，乙=55，所以 甲 最高。"}], "difficulty": "ood_logicchain"}
+{"messages": [{"role": "user", "content": "一件商品原价 120 元，打 -10 折出售，小明买了 8 件，一共花了多少钱？"}, {"role": "assistant", "content": "折后单价 = 120 × 0.8 = 96 元，共 96 × 8 = 768 元。"}], "difficulty": "ood_word3step"}
+{"messages": [{"role": "user", "content": "38 的平方是多少？"}, {"role": "assistant", "content": "38² = 38 × 38 = 1444。"}], "difficulty": "ood_square"}
+{"messages": [{"role": "user", "content": "乙 比 丁 高，丁 比 戊 低，戊 比 甲 低。四人中谁最高？"}, {"role": "assistant", "content": "由条件可得：乙=80，丁=13，戊=19，甲=32，所以 乙 最高。"}], "difficulty": "ood_logicchain"}

pondernet/demo_minicpm5.py

@@ -0,0 +1,82 @@
+# -*- coding: utf-8 -*-
+"""
+demo_minicpm5.py — 用真实的 MiniCPM5-2B 权重体验 PonderNet 循环思考推理
+
+前置: 已按 openbmb/MiniCPM5-2B 的 README 下载权重 (或直接指向 HF repo id,
+首次运行会自动下载约 5GB 权重)。CPU/GPU 均可。
+
+用法:
+    python demo_minicpm5.py --model /path/to/MiniCPM5-2B-cpu
+    python demo_minicpm5.py --model tchbcb/MiniCPM5-2B-cpu --signal drift --max-steps 6
+"""
+import argparse
+import time
+
+import torch
+from transformers import AutoTokenizer
+
+from ponder_llama import PonderLlamaForCausalLM, PonderLlamaConfig, LlamaConfig
+
+
+def build_model(path: str, signal: str, max_steps: int, device: str):
+    cfg = PonderLlamaConfig.from_llama(
+        LlamaConfig.from_pretrained(path),
+        ponder_signal=signal,          # entropy | msp | drift | learned
+        max_ponder_steps=max_steps,    # 每个位置最多思考几步
+        ponder_epsilon=0.01,           # 累计停机概率达 0.99 即停
+        ponder_start_layer=None,       # 默认最后 8 层作为"思考块"
+        ponder_window=1,               # prefill 时只对最后一个位置(即回答前)思考
+        commit_kv=True,                # 用思考后的混合表示写 KV (严格正确模式)
+    )
+    model = PonderLlamaForCausalLM.from_pretrained(
+        path, config=cfg, torch_dtype=torch.bfloat16 if device == "cuda" else torch.float32,
+        low_cpu_mem_usage=True,
+    ).to(device).eval()
+    return model
+
+
+def chat_generate(model, tokenizer, user_msg: str, device: str, max_new_tokens=256):
+    prompt = tokenizer.apply_chat_template(
+        [{"role": "user", "content": user_msg}],
+        tokenize=True, add_generation_prompt=True, return_tensors="pt",
+    ).to(device)
+    attn = torch.ones_like(prompt)
+    t0 = time.time()
+    out = model.generate(prompt, attention_mask=attn, max_new_tokens=max_new_tokens,
+                         do_sample=False)
+    dt = time.time() - t0
+    text = tokenizer.decode(out[0, prompt.shape[1]:], skip_special_tokens=True)
+
+    decodes = [e for e in model._ponder_log if e["mode"] == "decode"]
+    prefill = [e for e in model._ponder_log if e["mode"] == "prefill"]
+    steps = [e["steps_max"] for e in decodes]
+    print(f"\n[思考块 = 第{model.config.ponder_start_layer}~{model.config.num_hidden_layers-1}层 | "
+          f"信号={model.config.ponder_signal} | 上限={model.config.max_ponder_steps}步]")
+    print(f"[prefill 思考步数: {prefill[-1]['steps_max'] if prefill else '-'} | "
+          f"decode 平均思考 {sum(steps)/max(len(steps),1):.2f} 步 (共{len(steps)} token, "
+          f"{dt:.1f}s, {len(steps)/dt:.2f} tok/s)]")
+    return text
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", default="/home/z/my-project/models/MiniCPM5-2B-cpu")
+    ap.add_argument("--signal", default="entropy", choices=["entropy", "msp", "drift", "learned"])
+    ap.add_argument("--max-steps", type=int, default=8)
+    ap.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")
+    args = ap.parse_args()
+
+    tokenizer = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
+    model = build_model(args.model, args.signal, args.max_steps, args.device)
+    n_think = sum(p.numel() for p in model.ponder_head.parameters())
+    print(f"模型加载完毕 (新增 ponder 参数 {n_think} 个)")
+
+    for q in ["用一句话介绍你自己。",
+              "9.11 和 9.9 哪个大？请一步步说明。"]:
+        print("=" * 60)
+        print(f"Q: {q}")
+        print(f"A: {chat_generate(model, tokenizer, q, args.device)}")
+
+
+if __name__ == "__main__":
+    main()

pondernet/eval_ponder.py

@@ -0,0 +1,185 @@
+# -*- coding: utf-8 -*-
+"""
+eval_ponder.py — PonderNet 微调效果评估
+
+在难度分级评估集 (easy/medium/hard, 每行带 "difficulty" 字段) 上统计:
+  1. 全局 CE loss (纯语言建模损失, 不含 KL)
+  2. 各难度桶的平均思考步数 + 步数分布直方图
+     → 训练成功后应看到 hard > medium > easy 的步数分化
+  3. --max-steps 1 作为 baseline (等价关闭循环思考), 对比自适应推理
+  4. --generate: 对几个示例问题实际生成, 观察 decode 阶段每 token 思考步数
+
+用法 (训练前 baseline):
+  python eval_ponder.py --model MODEL_DIR --data data/mix.eval.jsonl \
+      --max-steps 8 --head-bias-init -1 --tag before
+
+用法 (训练后, 加载训好的 head + LoRA):
+  python eval_ponder.py --model MODEL_DIR --data data/mix.eval.jsonl \
+      --max-steps 8 --head out/ponder_lora/ponder_head.safetensors \
+      --adapter out/ponder_lora --tag after --generate
+"""
+from __future__ import annotations
+
+import argparse
+import time
+
+import torch
+import torch.nn.functional as F
+
+from transformers import AutoTokenizer
+
+from ponder_llama import PonderLlamaForCausalLM
+from train_ponder_head import load_jsonl, PonderDataset, collate
+
+
+@torch.no_grad()
+def run_eval(model, tok, rows, device, max_len, batch_size, tag):
+    ds = PonderDataset(rows, tok, max_len=max_len)
+    loader = torch.utils.data.DataLoader(
+        ds, batch_size=batch_size, shuffle=False,
+        collate_fn=lambda b: collate(b, tok.pad_token_id or 1))
+    K = model.config.max_ponder_steps
+
+    tot_nll, tot_tok = 0.0, 0
+    bucket = {}          # difficulty -> [n, steps_sum, step_hist{1..K}]
+    model.eval()
+    t0 = time.time()
+    offset = 0           # 全局样本偏移 (shuffle=False, rows 与 loader 顺序一致)
+    for input_ids, labels, attn in loader:
+        input_ids, labels, attn = input_ids.to(device), labels.to(device), attn.to(device)
+        out = model(input_ids=input_ids, attention_mask=attn, labels=labels,
+                    output_ponder=True)
+        logits = out.logits.float()
+        shift_logits = logits[:, :-1, :].transpose(1, 2)          # (B,V,L-1)
+        shift_labels = labels[:, 1:]
+        mask = shift_labels != -100
+        nll = F.cross_entropy(shift_logits, shift_labels,
+                              ignore_index=-100, reduction="sum")
+        tot_nll += nll.item()
+        tot_tok += int(mask.sum())
+        steps = out.ponder_steps[:, :-1][mask].float()            # 与训练口径一致 (shift 对齐)
+        # 按 difficulty 聚合 (rows 与 ds.examples 顺序一致)
+        for b_idx in range(input_ids.size(0)):
+            m_b = mask[b_idx]
+            n_b = int(m_b.sum())
+            if n_b == 0:
+                continue
+            s_b = out.ponder_steps[b_idx, :-1][m_b].float()
+            d = rows[offset + b_idx].get("difficulty", "unknown")
+            rec = bucket.setdefault(d, [0, 0.0, [0] * (K + 1)])
+            rec[0] += 1
+            rec[1] += float(s_b.mean())
+            for v in s_b.tolist():
+                rec[2][min(int(round(v)), K)] += 1
+        offset += input_ids.size(0)
+
+    print(f"\n===== 评估 [{tag}]  K={K}  ({time.time()-t0:.0f}s) =====")
+    print(f"全局 CE = {tot_nll/max(tot_tok,1):.4f}  (tokens={tot_tok})")
+    order = ["easy", "medium", "hard", "unknown"]
+    hdr = f"{'难度':<8}{'样本':>6}{'平均步数':>10}   步数分布 (1..{K})"
+    print(hdr)
+    for d in order:
+        if d not in bucket:
+            continue
+        n, s_sum, hist = bucket[d]
+        bar = "  " + " ".join(f"{h/max(sum(hist),1)*100:4.0f}%" for h in hist[1:])
+        print(f"{d:<8}{n:>6}{s_sum/n:>10.3f}{bar}")
+    other = [d for d in bucket if d not in order]
+    for d in other:
+        n, s_sum, hist = bucket[d]
+        bar = "  " + " ".join(f"{h/max(sum(hist),1)*100:4.0f}%" for h in hist[1:])
+        print(f"{d:<8}{n:>6}{s_sum/n:>10.3f}{bar}")
+    return bucket
+
+
+@torch.no_grad()
+def gen_examples(model, tok, device, tag):
+    prompts = [("easy", "天空是什么颜色"),
+               ("easy", "把'苹果'翻译成英文"),
+               ("medium", "一辆汽车每小时行驶 60 千米，行驶 3 小时，一共行驶多少千米？"),
+               ("hard", "鸡兔同笼，从上面数有 10 个头，从下面数有 28 只脚，鸡和兔各有几只？")]
+    print(f"\n----- 生成示例 [{tag}] -----")
+    model.eval()
+    for diff, q in prompts:
+        msgs = [{"role": "user", "content": q}]
+        prompt = tok.apply_chat_template(msgs, tokenize=True, add_generation_prompt=True)
+        if hasattr(prompt, "ids"):
+            prompt = list(prompt.ids)
+        if hasattr(prompt, "input_ids"):
+            prompt = prompt["input_ids"]
+            if prompt and isinstance(prompt[0], list):
+                prompt = prompt[0]
+        prompt = torch.tensor([prompt], device=device)
+        attn = torch.ones_like(prompt)
+        model._ponder_log.clear()
+        out = model.generate(prompt, attention_mask=attn, max_new_tokens=96,
+                             do_sample=False, pad_token_id=tok.pad_token_id or tok.eos_token_id)
+        text = tok.decode(out[0][prompt.size(1):], skip_special_tokens=True)
+        dec = [e for e in model._ponder_log if e["mode"] == "decode"]
+        avg = sum(e["steps_mean"] for e in dec) / max(len(dec), 1)
+        print(f"[{diff}] Q: {q}")
+        print(f"      A: {text[:160]}")
+        print(f"      decode 平均思考 {avg:.2f} 步/token ({len(dec)} tokens)\n")
+
+
+def main():
+    ap = argparse.ArgumentParser(description="PonderNet 微调效果评估")
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--data", required=True)
+    ap.add_argument("--head", default=None, help="ponder_head.safetensors (训练产物)")
+    ap.add_argument("--adapter", default=None, help="LoRA 适配器目录 (训练产物)")
+    ap.add_argument("--head-bias-init", type=float, default=None,
+                    help="未训练 head 的 bias 覆盖 (训练前 baseline 用 -1 与训练初始一致)")
+    ap.add_argument("--max-steps", type=int, default=8, help="思考步数上限 (1=baseline)")
+    ap.add_argument("--signal", default="learned")
+    ap.add_argument("--max-len", type=int, default=512)
+    ap.add_argument("--batch-size", type=int, default=4)
+    ap.add_argument("--tag", default="eval")
+    ap.add_argument("--generate", action="store_true")
+    ap.add_argument("--limit", type=int, default=None, help="只评前 N 条 (调试)")
+    ap.add_argument("--dtype", default="auto")
+    args = ap.parse_args()
+
+    device = "cuda" if torch.cuda.is_available() else "cpu"
+    dtype = (torch.bfloat16 if torch.cuda.is_bf16_supported() else torch.float16) \
+        if device == "cuda" and args.dtype == "auto" else getattr(torch, args.dtype)
+
+    rows = load_jsonl(args.data)
+    if args.limit:
+        rows = rows[: args.limit]
+    print(f"评估样本: {len(rows)} 条 | K={args.max_steps} | tag={args.tag}")
+
+    model = PonderLlamaForCausalLM.from_ponder(
+        args.model, ponder_kwargs={"ponder_signal": args.signal,
+                                   "max_ponder_steps": args.max_steps,
+                                   "ponder_all_positions": True},
+        torch_dtype=dtype, attn_implementation="sdpa")
+    model.ponder_head = model.ponder_head.float()
+    if args.head_bias_init is not None:
+        with torch.no_grad():
+            model.ponder_head.bias.fill_(args.head_bias_init)
+    if args.head:
+        from safetensors.torch import load_file
+        sd = {k.replace("ponder_head.", ""): v
+              for k, v in load_file(args.head).items()}
+        model.ponder_head.load_state_dict(
+            {k: v for k, v in sd.items() if k in ("weight", "bias")})
+        print(f"已加载 head: {args.head}")
+    if args.adapter:
+        from peft import PeftModel
+        model = PeftModel.from_pretrained(model, args.adapter)
+        print(f"已加载 LoRA: {args.adapter}")
+    model.to(device)
+
+    if args.head is None and args.head_bias_init is None and args.max_steps > 1:
+        print("提示: 未加载 head 且未设 --head-bias-init, 使用出厂初始化 (bias=-4)")
+
+    run_eval(model, AutoTokenizer.from_pretrained(args.model, trust_remote_code=True),
+             rows, device, args.max_len, args.batch_size, args.tag)
+    if args.generate:
+        gen_examples(model, AutoTokenizer.from_pretrained(args.model, trust_remote_code=True),
+                     device, args.tag)
+
+
+if __name__ == "__main__":
+    main()

pondernet/out_t4/ponder_head.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c25dbffb159717cc4894e232067ec4a06bd16cae220ef7b1b60fa49d99915404
+size 8412

pondernet/ponder_llama.py

@@ -0,0 +1,434 @@
+# -*- coding: utf-8 -*-
+"""
+ponder_llama.py — 给 transformers 5.x 原生 LlamaForCausalLM 加上 PonderNet 式
+"循环思考 + 自适应停机" 推理。可直接加载 tchbcb/MiniCPM5-2B-cpu (即 openbmb/MiniCPM5-2B,
+标准 LLaMA 架构, LlamaForCausalLM) 的权重, 新增参数只有 halting 头 (hidden_size -> 1)。
+
+核心思想 (PonderNet, Banino et al. 2021):
+  把解码层切成两段: 前段 [0, ponder_start_layer) 正常跑一遍;
+  后段 [ponder_start_layer, L) 作为"思考块 (ponder block)"循环执行, 每迭代一步:
+      h_k  = block(h_{k-1})
+      lam_k = halting(h_k)              # 停机概率, 每个位置独立
+      w_k   = remaining * lam_k         # 本步停下的概率质量
+      mix  += w_k * h_k                 # 概率加权混合输出
+      remaining *= (1 - lam_k)
+  直到累计质量 >= 1 - epsilon 或到达 max_ponder_steps; 剩余质量全部记给最后一步。
+
+KV cache 策略 (保证 generate() 兼容):
+  思考块每次迭代都会经 past_key_values.update() 写入当前 token 的 KV,
+  迭代之间对思考块各层执行 layer.crop(-W) 撤销写入; 循环结束后用"混合表示"
+  h_mix 做一次 commit pass 写入最终 KV —— 后续 token 注意到的正是思考后的表示。
+  第 1 步就停机的位置无需 commit (cache 即为该表示的 KV), 自动跳过以省计算。
+
+Halting 信号 (无需训练即可用的推理启发式):
+  - "entropy": lam = 1 - 归一化熵 (模型越自信越停)         [默认]
+  - "msp":     lam = max softmax 概率
+  - "drift":   相邻两次迭代 hidden 余弦相似度超过阈值 -> 已收敛 -> 停
+  - "learned": 真 PonderNet 线性停机头 (随机初始化无意义, 必须训练;
+               训练用下方 ponder loss = CE + beta * KL(w || Geometric(p_g)))
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from transformers import AutoConfig, AutoModelForCausalLM
+from transformers.cache_utils import DynamicCache
+from transformers.masking_utils import create_causal_mask
+from transformers.modeling_outputs import CausalLMOutputWithPast
+from transformers.models.llama.configuration_llama import LlamaConfig
+from transformers.models.llama.modeling_llama import LlamaForCausalLM
+from transformers.utils import can_return_tuple
+
+
+# --------------------------------------------------------------------------
+# Config
+# --------------------------------------------------------------------------
+class PonderLlamaConfig(LlamaConfig):
+    """在 LlamaConfig 之上追加 PonderNet 超参 (其余字段与原 Llama config 完全兼容)."""
+
+    model_type = "llama_ponder"
+
+    def __init__(self, **kwargs):
+        self.ponder_start_layer = kwargs.pop("ponder_start_layer", None)  # None = 最后 8 层
+        self.ponder_window = kwargs.pop("ponder_window", 1)               # prefill 思考窗口 W
+        self.max_ponder_steps = kwargs.pop("max_ponder_steps", 8)         # 思考块最多循环步数 K
+        self.ponder_epsilon = kwargs.pop("ponder_epsilon", 0.01)          # 累计概率 1-eps 即停
+        self.ponder_signal = kwargs.pop("ponder_signal", "entropy")       # entropy|msp|drift|learned
+        self.ponder_all_positions = kwargs.pop("ponder_all_positions", False)  # 全序列位置思考
+        self.commit_kv = kwargs.pop("commit_kv", True)                    # 用混合表示写最终 KV
+        self.ponder_prior_p = kwargs.pop("ponder_prior_p", 0.5)           # 几何先验 p_g (训练)
+        self.ponder_loss_beta = kwargs.pop("ponder_loss_beta", 0.01)      # KL 正则权重 beta
+        self.drift_threshold = kwargs.pop("drift_threshold", 0.98)        # drift 相似度阈值
+        super().__init__(**kwargs)
+        if self.ponder_start_layer is None:
+            self.ponder_start_layer = max(0, self.num_hidden_layers - 8)
+
+    @classmethod
+    def from_llama(cls, llama_config: LlamaConfig, **ponder_kwargs) -> "PonderLlamaConfig":
+        """从标准 LlamaConfig 派生 (保留全部架构字段, 丢弃非配置项)。
+        兼容两种来源: 标准 llama config.json (无 ponder 字段) 与本类保存的
+        llama_ponder config.json (含 ponder 字段) —— 显式 ponder_kwargs 优先覆盖。"""
+        d = llama_config.to_dict()
+        for k in ("model_type", "architectures", "_name_or_path", "transformers_version",
+                  "torch_dtype"):
+            d.pop(k, None)
+        ponder_keys = [k for k in d
+                       if "ponder" in k or k in ("commit_kv", "drift_threshold")]
+        for k in ponder_keys:
+            if k in ponder_kwargs:      # 显式参数覆盖 config 里的旧值, 避免重复 kwargs 报错
+                d.pop(k)
+        return cls(**d, **ponder_kwargs)
+
+
+# --------------------------------------------------------------------------
+# 输出对象
+# --------------------------------------------------------------------------
+@dataclass
+class PonderCausalLMOutputWithPast(CausalLMOutputWithPast):
+    """额外带出 ponder 诊断信息."""
+    ponder_weights: torch.Tensor | None = None   # (B, W, K) 每步停机质量分布
+    ponder_steps: torch.Tensor | None = None     # (B, W) 每位置实际思考步数
+    ponder_kl: torch.Tensor | None = None        # 标量, KL(w‖Geometric) 正则项 (labels 时)
+
+
+# --------------------------------------------------------------------------
+# Model
+# --------------------------------------------------------------------------
+class PonderLlamaForCausalLM(LlamaForCausalLM):
+    """
+    用法 A — 直接加载 MiniCPM5-2B 权重 (键名与 LlamaForCausalLM 完全一致):
+        from ponder_llama import PonderLlamaForCausalLM, PonderLlamaConfig
+        base = LlamaConfig.from_pretrained("tchbcb/MiniCPM5-2B-cpu")
+        cfg  = PonderLlamaConfig.from_llama(base)
+        model = PonderLlamaForCausalLM.from_pretrained(
+            "tchbcb/MiniCPM5-2B-cpu", config=cfg, torch_dtype="bfloat16")
+        # missing keys 只有 ponder_head (仅 signal="learned" 时参与前向, 其余信号忽略)
+
+    用法 B — 一行加载:
+        model = PonderLlamaForCausalLM.from_ponder("tchbcb/MiniCPM5-2B-cpu")
+
+    generate() / chat 流程与 HF 标准完全一致; model.ponder_log 记录每次 forward 的思考步数。
+    """
+
+    def __init__(self, config: PonderLlamaConfig):
+        super().__init__(config)
+        cfg = self.config
+        # 真 PonderNet 停机头; bias=-4 使初始 lam≈0.018 (初始倾向于多想几步)
+        self.ponder_head = nn.Linear(cfg.hidden_size, 1, bias=True)
+        nn.init.zeros_(self.ponder_head.weight)
+        nn.init.constant_(self.ponder_head.bias, -4.0)
+        self._ponder_log: list[dict] = []   # 诊断日志 (每次 forward 追加, 最多保留 4096 条)
+        self.post_init()
+
+    @classmethod
+    def from_ponder(cls, pretrained_path: str, ponder_kwargs: dict | None = None, **kw):
+        """从标准 Llama 权重目录 / HF repo 一行构造 Ponder 模型."""
+        base = LlamaConfig.from_pretrained(pretrained_path)
+        cfg = PonderLlamaConfig.from_llama(base, **(ponder_kwargs or {}))
+        return cls.from_pretrained(pretrained_path, config=cfg, **kw)
+
+    # ------------------------------------------------------------------ #
+    # 内部工具
+    # ------------------------------------------------------------------ #
+    @property
+    def _ponder_layers(self):
+        s = self.config.ponder_start_layer
+        return self.model.layers[s: self.config.num_hidden_layers]
+
+    def _crop_ponder_kv(self, cache, n_remove: int):
+        """只回滚思考块层的 KV 写入 (前段层的 KV 不许动)."""
+        if cache is None or n_remove <= 0:
+            return
+        s = self.config.ponder_start_layer
+        for li in range(s, self.config.num_hidden_layers):
+            cache.layers[li].crop(-n_remove)
+
+    def _run_block(self, h, attention_mask, position_ids, position_embeddings,
+                   cache, kwargs):
+        """跑一遍思考块 (最后若干层).
+        训练且开启 gradient checkpointing 时逐层重算省激活 —— 思考块最多循环 K 次,
+        激活占用是普通前向的 K 倍, checkpointing 是小显存卡 (如 T4 16GB) 的关键."""
+        ckpt_fn = getattr(self.model, "_gradient_checkpointing_func", None)
+        use_ckpt = (self.training and cache is None
+                    and getattr(self.model, "gradient_checkpointing", False)
+                    and ckpt_fn is not None)
+        for layer in self._ponder_layers:
+            if use_ckpt:
+                h = ckpt_fn(
+                    layer.__call__, h,
+                    attention_mask=attention_mask,
+                    position_ids=position_ids,
+                    position_embeddings=position_embeddings,
+                    past_key_values=None,
+                    use_cache=False,
+                    **kwargs,
+                )
+            else:
+                h = layer(
+                    h,
+                    attention_mask=attention_mask,
+                    position_ids=position_ids,
+                    position_embeddings=position_embeddings,
+                    past_key_values=cache,
+                    use_cache=cache is not None,
+                    **kwargs,
+                )
+        return h
+
+    def _halting_prob(self, h_window, h_prev_window) -> torch.Tensor:
+        """返回 (B, W) 的停机概率 lam ∈ (0, 1)."""
+        cfg = self.config
+        B, W = h_window.shape[:2]
+        if cfg.ponder_signal == "learned":
+            # ponder_head 恒为 fp32 (数值稳定), 输入 cast 到其权重 dtype,
+            # 否则半精度模型 (fp16/bf16) 下 Linear 会报 dtype 不匹配错误
+            lam = torch.sigmoid(
+                self.ponder_head(h_window.to(self.ponder_head.weight.dtype))
+            ).squeeze(-1)
+        elif cfg.ponder_signal in ("entropy", "msp"):
+            logits = self.lm_head(self.model.norm(h_window)).float()
+            probs = F.softmax(logits, dim=-1)
+            if cfg.ponder_signal == "msp":
+                lam = probs.max(dim=-1).values
+            else:  # entropy: 自信 -> 熵小 -> lam 大
+                ent = -(probs * torch.log(probs.clamp_min(1e-9))).sum(-1)
+                lam = 1.0 - ent / torch.log(torch.tensor(float(logits.size(-1)), device=logits.device))
+        elif cfg.ponder_signal == "drift":
+            if h_prev_window is None:      # 第一步没有参照, 强制继续思考
+                lam = torch.zeros(B, W, device=h_window.device, dtype=h_window.dtype)
+            else:
+                cos = F.cosine_similarity(h_window.float(), h_prev_window.float(), dim=-1)
+                tau = cfg.drift_threshold
+                lam = (cos - tau) / (1.0 - tau + 1e-6)   # cos 越接近 1 (不再变化) 越停
+        else:
+            raise ValueError(f"unknown ponder_signal: {cfg.ponder_signal}")
+        # 统一 cast 回隐状态 dtype: 保证 remaining/mix/w_stack 混合链路 dtype 一致,
+        # 否则 fp32 的 lam 会把整条链路升成 fp32, 之后 norm/lm_head (half 权重) 会崩
+        return lam.to(h_window.dtype).clamp(1e-6, 1.0 - 1e-6)
+
+    def _win_mask(self, W, past_len, device, dtype, attention_mask=None):
+        """窗口迭代的加性因果 mask: keys = [0..past_len+W), queries = 窗口 W 个位置。
+
+        必须手工构建而不能用 create_causal_mask: 后者从非思考层读取缓存长度,
+        而思考块层已被 crop 到 past_len, 会得到多 W 的错误 kv 尺寸。
+        返回 float 加性 mask (sdpa / eager 通用)。"""
+        total = past_len + W
+        q_idx = torch.arange(past_len, total, device=device)
+        k_idx = torch.arange(total, device=device)
+        allowed = q_idx.unsqueeze(-1) >= k_idx.unsqueeze(0)              # (W, total)
+        if attention_mask is not None:
+            allowed = allowed.unsqueeze(0) & attention_mask[:, :total].to(torch.bool).unsqueeze(1)
+            m = torch.zeros(allowed.size(0), 1, W, total, dtype=dtype, device=device)
+            m = m.masked_fill(~allowed.unsqueeze(1), torch.finfo(dtype).min)   # (B,1,W,total)
+        else:
+            m = torch.zeros(1, 1, W, total, dtype=dtype, device=device)
+            m = m.masked_fill(~allowed.unsqueeze(0).unsqueeze(0), torch.finfo(dtype).min)
+        return m
+
+    def _log(self, entry: dict):
+        self._ponder_log.append(entry)
+        if len(self._ponder_log) > 4096:
+            del self._ponder_log[: len(self._ponder_log) - 4096]
+
+    # ------------------------------------------------------------------ #
+    # 主前向
+    # ------------------------------------------------------------------ #
+    @can_return_tuple
+    def forward(
+        self,
+        input_ids: torch.LongTensor | None = None,
+        attention_mask: torch.Tensor | None = None,
+        position_ids: torch.LongTensor | None = None,
+        past_key_values: DynamicCache | None = None,
+        inputs_embeds: torch.FloatTensor | None = None,
+        labels: torch.LongTensor | None = None,
+        use_cache: bool | None = None,
+        logits_to_keep: int | torch.Tensor = 0,
+        output_ponder: bool = False,
+        **kwargs,
+    ) -> PonderCausalLMOutputWithPast:
+        cfg = self.config
+        m = self.model
+
+        # ---- 输入 & 位置 ------------------------------------------------
+        if (input_ids is None) ^ (inputs_embeds is not None):
+            raise ValueError("You must specify exactly one of input_ids or inputs_embeds")
+        if inputs_embeds is None:
+            inputs_embeds = m.embed_tokens(input_ids)
+        B, Q = inputs_embeds.shape[:2]
+
+        if use_cache is None:
+            use_cache = False if self.training else cfg.use_cache
+        if use_cache and past_key_values is None:
+            past_key_values = DynamicCache(config=cfg)
+
+        cache_position = kwargs.pop("cache_position", None)
+        past_seen = past_key_values.get_seq_length() if past_key_values is not None else 0
+        if position_ids is None:
+            base = (cache_position if cache_position is not None
+                    else torch.arange(Q, device=inputs_embeds.device) + past_seen)
+            position_ids = base.reshape(1, -1).expand(B, -1)[:, :Q].to(inputs_embeds.device)
+
+        # ---- mask / rope (整次 forward 内复用) ---------------------------
+        causal_mask = create_causal_mask(
+            config=cfg, inputs_embeds=inputs_embeds, attention_mask=attention_mask,
+            past_key_values=past_key_values, position_ids=position_ids,
+        )
+        position_embeddings = m.rotary_emb(inputs_embeds, position_ids=position_ids)
+
+        # ---- Stage A: 前段层正常单遍 -------------------------------------
+        h = inputs_embeds
+        for layer in m.layers[: cfg.ponder_start_layer]:
+            h = layer(
+                h,
+                attention_mask=causal_mask,
+                position_ids=position_ids,
+                position_embeddings=position_embeddings,
+                past_key_values=past_key_values,
+                use_cache=use_cache,
+                **kwargs,
+            )
+
+        # ---- 思考窗口 W --------------------------------------------------
+        # decode (Q==1) 恒为 1; 训练/显式开启时对全部位置; 其余 prefill 思考末尾 W 个位置
+        if Q == 1 or self.training or cfg.ponder_all_positions:
+            W = Q
+        else:
+            W = max(1, min(cfg.ponder_window, Q))
+
+        eps = cfg.ponder_epsilon
+        K = cfg.max_ponder_steps
+        device, dtype = inputs_embeds.device, inputs_embeds.dtype
+
+        remaining = torch.ones(B, W, device=device, dtype=dtype)
+        cum = torch.zeros(B, W, device=device, dtype=dtype)
+        active = torch.ones(B, W, device=device, dtype=torch.bool)
+        mix = torch.zeros(B, W, h.size(-1), device=device, dtype=dtype)
+        w_list: list[torch.Tensor] = []
+        h_prev_win = None
+        h_full = None            # 第 1 次迭代的全序列输出 (W<Q 时非窗口位置沿用)
+        h_win = h[:, -W:, :] if W < Q else h
+        executed = 0
+        win_mask_built = False   # 窗口迭代专用 mask (懒构建一次)
+
+        for k in range(1, K + 1):
+            executed = k
+            if k > 1:
+                # 替换式写入: 先回滚上一次迭代在思考块层的 KV, 本轮 update 再补回,
+                # 使任意时刻 cache 长度恒等于真实 token 数 (对 W==Q 同样必需!)
+                self._crop_ponder_kv(past_key_values, W)
+            # 仅当有 cache 且 W<Q 时才做窗口局部重算; 无 cache (训练) 一律全序列重算
+            use_window = (k > 1) and (past_key_values is not None) and (W < Q)
+            if not use_window:
+                h = self._run_block(h, causal_mask, position_ids,
+                                    position_embeddings, past_key_values, kwargs)
+                h_win = h if W == Q else h[:, -W:, :]
+                if W < Q:
+                    h_full = h
+            else:
+                if not win_mask_built:
+                    past_len = int(past_key_values.layers[cfg.ponder_start_layer].get_seq_length())
+                    mask_win = self._win_mask(W, past_len, device, dtype, attention_mask)
+                    pos_emb_win = (position_embeddings[0][:, -W:],
+                                   position_embeddings[1][:, -W:])
+                    win_mask_built = True
+                h_win = self._run_block(h_win, mask_win, position_ids[:, -W:],
+                                        pos_emb_win, past_key_values, kwargs)
+
+            lam = self._halting_prob(h_win, h_prev_win)          # (B, W)
+            lam_eff = lam * active.to(lam.dtype)
+            w_k = remaining * lam_eff
+            mix = mix + w_k.unsqueeze(-1) * h_win
+            w_list.append(w_k)
+            remaining = remaining * (1.0 - lam_eff)
+            cum = cum + w_k
+            active = active & (cum < 1.0 - eps)
+            h_prev_win = h_win
+            if not bool(active.any()):
+                break
+
+        # 剩余概率质量全部记给最后一步 (PonderNet 的 p_N 截断)
+        mix = mix + remaining.unsqueeze(-1) * h_win
+        w_stack = torch.stack(w_list, dim=-1).to(dtype)          # (B, W, K_exec)
+        w_stack = torch.cat([w_stack[..., :-1],
+                             w_stack[..., -1:] + remaining.unsqueeze(-1)], dim=-1)
+
+        # ---- 组装最终 hidden --------------------------------------------
+        if W < Q:
+            h_final = h_full
+            h_final[:, -W:, :] = mix
+        else:
+            h_final = mix
+
+        # ---- commit pass: 用"思考后"的混合表示写最终 KV ------------------
+        if use_cache and cfg.commit_kv and past_key_values is not None and executed > 1:
+            self._crop_ponder_kv(past_key_values, W)
+            if not win_mask_built:   # W==Q 的场景
+                self._run_block(h_final, causal_mask, position_ids,
+                                position_embeddings, past_key_values, kwargs)
+            else:
+                self._run_block(h_final[:, -W:, :], mask_win, position_ids[:, -W:],
+                                pos_emb_win, past_key_values, kwargs)
+            # commit 输出丢弃, KV 由 mix 重新计算 —— 后续 token 看到的就是思考后的表示
+
+        # ---- logits / loss ----------------------------------------------
+        slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
+        logits = self.lm_head(m.norm(h_final[:, slice_indices, :]))
+
+        loss, ponder_kl = None, None
+        if labels is not None:
+            ce_logits = logits.float()
+            loss = F.cross_entropy(
+                ce_logits[..., :-1, :].reshape(-1, ce_logits.size(-1)),
+                labels[..., 1:].reshape(-1), ignore_index=-100)
+
+            # PonderNet 正则: KL(w || Geometric(p_g)), 截断到 K 步并归一化。
+            # 必须升到 fp32 再取 log: fp16 下 p=0 处 clamp_min(1e-8) 仍会下溢为 0,
+            # log(0)=-inf 与 p=0 相乘直接得到 NaN, 训练瞬间发散
+            p = (w_stack / w_stack.sum(-1, keepdim=True).clamp_min(1e-8)).float()  # (B,W,K)
+            Kd = p.size(-1)
+            n_idx = torch.arange(1, Kd + 1, device=p.device, dtype=p.dtype)
+            pg = cfg.ponder_prior_p
+            geom = (1 - pg) ** (n_idx - 1) * pg
+            geom = torch.cat([geom[..., :-1],
+                              ((1 - pg) ** (Kd - 1) * torch.ones_like(geom[..., -1:]))],
+                             dim=-1)
+            geom = geom / geom.sum()
+            kl = (p * (torch.log(p.clamp_min(1e-8))
+                       - torch.log(geom.expand_as(p)))).sum(-1)
+            ponder_kl = kl.mean()
+            loss = loss + cfg.ponder_loss_beta * ponder_kl
+
+        # ---- 诊断 --------------------------------------------------------
+        cum_w = torch.cumsum(w_stack, dim=-1)
+        steps_used = ((cum_w < 1.0 - eps).sum(-1) + 1).clamp(max=w_stack.size(-1))  # (B, W)
+        self._log({
+            "mode": "train" if self.training else ("decode" if Q == 1 else "prefill"),
+            "signal": cfg.ponder_signal,
+            "executed": executed,
+            "steps_max": int(steps_used.max()),
+            "steps_mean": float(steps_used.float().mean()),
+        })
+
+        return PonderCausalLMOutputWithPast(
+            loss=loss,
+            logits=logits,
+            past_key_values=past_key_values,
+            ponder_weights=w_stack if output_ponder else None,
+            ponder_steps=steps_used if output_ponder else None,
+            ponder_kl=ponder_kl,
+        )
+
+
+# --------------------------------------------------------------------------
+# AutoClass 注册 (可重复 import 安全)
+# --------------------------------------------------------------------------
+try:
+    AutoConfig.register("llama_ponder", PonderLlamaConfig)
+    AutoModelForCausalLM.register(PonderLlamaConfig, PonderLlamaForCausalLM)
+except Exception:
+    pass

pondernet/ponder_steps_demo.py

@@ -0,0 +1,55 @@
+# -*- coding: utf-8 -*-
+"""ponder_steps_demo.py — 可视化 halting 头学习过程中"平均思考步数"的漂移
+(tiny 模型 CPU, 展示 ponder loss 中 CE 与 KL 两股力量的联合作用)"""
+import torch
+from torch.utils.data import DataLoader
+
+from ponder_llama import PonderLlamaConfig, PonderLlamaForCausalLM
+from train_ponder_head import setup_trainables, train_one_epoch
+from test_train import make_args, synthetic_batches
+
+
+def main():
+    torch.manual_seed(3)
+    model = PonderLlamaForCausalLM(PonderLlamaConfig(
+        vocab_size=256, hidden_size=64, intermediate_size=128,
+        num_hidden_layers=6, num_attention_heads=4, num_key_value_heads=2,
+        head_dim=16, max_position_embeddings=512, bos_token_id=0,
+        eos_token_id=1, pad_token_id=1, ponder_start_layer=4,
+        max_ponder_steps=8, ponder_epsilon=0.01, ponder_signal="learned",
+        ponder_all_positions=True, ponder_prior_p=0.4, ponder_loss_beta=0.05))
+
+    args = make_args(train_mode="head+block", lr_block=2e-3, lr_head=5e-3,
+                     prior_p=0.4, beta=0.05)
+    model, groups, _ = setup_trainables(model, args)
+
+    data = synthetic_batches(n=8, seqlen=24)
+    loader = [(torch.cat([x[0] for x in data]),
+               torch.cat([x[1] for x in data]),
+               torch.cat([x[2] for x in data]))]
+    ids, labels, attn = loader[0]
+    opt = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: 1.0)
+
+    print(f"几何先验 p_g={args.prior_p} → 期望步数 ≈ 1/p_g ≈ {1/args.prior_p:.1f} "
+          f"(K 截断为 8)\n")
+    print(f"{'step':>6} | {'CE':>7} | {'KL':>7} | {'平均思考步数':>10}")
+    print("-" * 42)
+    for i in range(61):
+        gs = train_one_epoch(model, loader, opt, sched, "cpu", accum=1, log_every=0)
+        if i % 10 == 0:
+            with torch.no_grad():
+                out = model(input_ids=ids, attention_mask=attn, labels=labels,
+                            output_ponder=True)
+            ce = out.loss.item() - 0.05 * out.ponder_kl.item()
+            w = out.ponder_weights                    # (B, Q, K)
+            p = w / w.sum(-1, keepdim=True)
+            exp_steps = (p * torch.arange(1, p.size(-1) + 1)).sum(-1).mean().item()
+            print(f"{i:>6} | {ce:>7.4f} | {out.ponder_kl.item():>7.4f} | {exp_steps:>10.2f}")
+    print("-" * 42)
+    print("观察: CE 持续下降; 平均思考步数从初始值向几何先验期望 (1/p_g) 靠拢,")
+    print("同时保留对'哪些位置值得多想'的区分 —— 这正是 PonderNet 的学习目标。")
+
+
+if __name__ == "__main__":
+    main()

pondernet/run_t4.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+# ==========================================================================
+# run_t4.sh — T4 (16GB) 一键训练 + 评估 PonderNet halting 头
+#
+# 流程: 环境自检 → 依赖检查 → 权重下载(约5GB, 磁盘不足自动清理) →
+#       生成难度分级数据 → head+lora 训练(fp16) → 训前/训后评估 + 思考收益
+#
+# 用法:  bash run_t4.sh
+# 可用环境变量覆盖默认值:
+#   MODEL_DIR  模型权重目录 (默认 /home/z/my-project/models/MiniCPM5-2B-cpu)
+#   HF_TOKEN   HuggingFace token
+#   EPOCHS     训练轮数 (默认 2)
+# ==========================================================================
+set -euo pipefail
+cd "$(dirname "$0")"
+
+MODEL_DIR="${MODEL_DIR:-/home/z/my-project/models/MiniCPM5-2B-cpu}"
+# 安全: token 从环境变量读取, 绝不硬编码 (HF 上传时会被秘密检测拦截)
+: "${HF_TOKEN:?请先 export HF_TOKEN=<你的 huggingface token>}"; export HF_TOKEN
+EPOCHS="${EPOCHS:-2}"
+OUT=out/ponder-t4
+
+step() { echo -e "\n\033[1;36m========== $1 ==========\033[0m"; }
+
+step "0/5 环境自检 (必须已挂载 T4)"
+python - <<'EOF'
+import torch
+assert torch.cuda.is_available(), "未检测到 GPU! 请先挂载 T4 再运行本脚本"
+name = torch.cuda.get_device_name(0)
+major, minor = torch.cuda.get_device_capability(0)
+mem = torch.cuda.get_device_properties(0).total_memory / 1e9
+print(f"GPU: {name}  sm_{major}{minor}  {mem:.1f} GB")
+print(f"bf16 支持: {torch.cuda.is_bf16_supported()}  (T4=False → 自动 fp16 + GradScaler)")
+EOF
+df -h . | tail -1
+
+step "1/5 依赖检查"
+python -c "import transformers, safetensors, peft" 2>/dev/null || {
+    echo "安装缺失依赖..."
+    python -m pip install -q peft safetensors "transformers>=5.6.2"
+}
+python -c "import transformers, peft; print(f'transformers {transformers.__version__}, peft {peft.__version__}')"
+
+step "2/5 权重检查/下载 (约 5.0 GB)"
+if [ -f "$MODEL_DIR/model-00000-of-00001.safetensors" ]; then
+    echo "权重已就绪: $MODEL_DIR"
+else
+    FREE_GB=$(df --output=avail -BG . | tail -1 | tr -dc '0-9')
+    echo "可用磁盘: ${FREE_GB}GB (需要约 6GB)"
+    if [ "${FREE_GB:-0}" -lt 6 ]; then
+        echo "磁盘不足, 自动清理缓存 (pip/puppeteer/playwright)..."
+        python -m pip cache purge 2>/dev/null || true
+        rm -rf ~/.cache/puppeteer ~/.cache/ms-playwright 2>/dev/null || true
+        df -h . | tail -1
+    fi
+    python - "$MODEL_DIR" "$HF_TOKEN" <<'EOF'
+import sys
+from huggingface_hub import snapshot_download
+snapshot_download("tchbcb/MiniCPM5-2B-cpu", local_dir=sys.argv[1],
+                  token=sys.argv[2],
+                  allow_patterns=["*.safetensors", "*.json", "tokenizer*"],
+                  max_workers=4)
+print("权重下载完成")
+EOF
+fi
+
+step "3/5 生成难度分级数据 (easy/medium/hard)"
+python train_ponder_head.py --make-mix-data data/pondernet_train.jsonl
+
+step "4/5 训练: head+lora (fp16 + 梯度重算, T4 推荐配置)"
+python train_ponder_head.py --model "$MODEL_DIR" \
+    --data data/pondernet_train.jsonl \
+    --train-mode head+lora --lora-scope ponder --lora-r 16 \
+    --dtype auto --grad-ckpt \
+    --epochs "$EPOCHS" --batch-size 2 --accum 8 --max-len 384 \
+    --max-steps 8 --epsilon 0.01 --prior-p 0.4 --beta 0.05 --head-bias-init -1.0 \
+    --lr-head 1e-3 --lr-lora 2e-4 \
+    --log-every 5 --output "$OUT"
+
+step "5/5 评估: 训前 vs 训后 + 思考收益 (K=8 vs K=1)"
+python eval_ponder.py --model "$MODEL_DIR" \
+    --head "$OUT/ponder_head.safetensors" --adapter "$OUT" \
+    --data data/pondernet_train.eval.jsonl \
+    --compare --k1-baseline --batch-size 4 \
+    --max-steps 8 --epsilon 0.01 --prior-p 0.4 --beta 0.05
+
+echo -e "\n\033[1;32m========== 全部完成 ==========\033[0m"
+echo "产物目录: $OUT"
+echo "  - ponder_head.safetensors   (训练后的 halting 头)"
+echo "  - adapter_model.safetensors (思考块 LoRA)"
+echo "评估结论看最后一组表格: 各难度桶 avg_steps 是否分化 (easy<hard), 以及思考收益 ΔCE"

pondernet/test_gpu_readiness.py

@@ -0,0 +1,136 @@
+# -*- coding: utf-8 -*-
+"""test_gpu_readiness.py — 真实 GPU (T4/fp16) 关键路径的 CPU 预验证
+
+T4 是 pre-Ampere 卡: 无 bf16, 只能 fp16 + GradScaler。以下场景在 CPU 上用 fp16
+提前复现, 全部通过后才可能在 T4 上一次跑通:
+  [1] learned 信号: fp16 模型 + fp32 ponder_head 的 dtype 转换链 (修复前必崩)
+  [2] fp16 下 KL 正则不 NaN: 大 λ → remaining 下溢 0 → w_stack 含 0 → p*log(p)
+      在 fp32 域计算 (修复前 fp16 直接 NaN)
+  [3] entropy/msp 信号: fp16 前向 (eval 场景), lam 统一 cast 回序列 dtype
+  [4] 思考块 gradient checkpointing: 开/关 loss 一致, 反传正常
+  [5] train_one_epoch + GradScaler 集成路径 (disabled scaler = 普通 step)
+"""
+import torch
+
+from ponder_llama import PonderLlamaConfig, PonderLlamaForCausalLM
+from train_ponder_head import setup_trainables, train_one_epoch
+
+
+def tiny_cfg(**kw):
+    d = dict(vocab_size=256, hidden_size=64, intermediate_size=128,
+             num_hidden_layers=6, num_attention_heads=4, num_key_value_heads=2,
+             head_dim=16, max_position_embeddings=512, bos_token_id=0,
+             eos_token_id=1, pad_token_id=1,
+             ponder_start_layer=4, max_ponder_steps=5, ponder_epsilon=0.01,
+             ponder_signal="learned", ponder_all_positions=True,
+             ponder_prior_p=0.4, ponder_loss_beta=0.05)
+    d.update(kw)
+    return PonderLlamaConfig(**d)
+
+
+def batch(vocab=256, B=2, S=24, seed=0):
+    g = torch.Generator().manual_seed(seed)
+    ids = torch.randint(3, vocab, (B, S), generator=g)
+    return (ids, ids.clone(), torch.ones(B, S, dtype=torch.long))
+
+
+def main():
+    torch.manual_seed(0)
+    ids, labels, attn = batch()
+
+    # ---------- [1] learned: fp16 模型 + fp32 head ----------
+    m = PonderLlamaForCausalLM(tiny_cfg())
+    m = m.to(torch.float16)          # 模拟 T4 fp16 权重
+    m.ponder_head = m.ponder_head.float()
+    m.eval()
+    out = m(input_ids=ids, attention_mask=attn, labels=labels, output_ponder=True)
+    assert torch.isfinite(out.loss), f"[1] loss 非有限: {out.loss}"
+    assert out.logits.dtype == torch.float16
+    out.loss.backward()              # 反传穿过 fp16 图到 fp32 head
+    g = m.ponder_head.weight.grad
+    assert g is not None and torch.isfinite(g).all() and g.abs().sum() > 0
+    print(f"[1] fp16 learned 前向+反传 OK  loss={out.loss.item():.4f}  "
+          f"head.grad|.|mean={g.abs().mean().item():.2e}")
+
+    # ---------- [2] fp16 KL NaN 防护: 大 λ 1 步停机 + 小 λ 跑满, KL 均须有限 ----------
+    torch.manual_seed(1)
+    m = PonderLlamaForCausalLM(tiny_cfg())
+    m = m.to(torch.float16)
+    m.ponder_head = m.ponder_head.float()
+    with torch.no_grad():
+        m.ponder_head.bias.fill_(6.0)    # sigmoid(6)≈0.9975 → 第 1 步 cum 过线即停
+        m.ponder_head.weight.zero_()
+    m.train()
+    out = m(input_ids=ids, attention_mask=attn, labels=labels, output_ponder=True)
+    assert out.ponder_kl is not None and torch.isfinite(out.ponder_kl), \
+        f"[2] fp16 KL NaN/Inf (大λ): {out.ponder_kl}"
+    assert torch.isfinite(out.loss)
+    assert torch.isfinite(out.ponder_weights).all()
+    print(f"[2a] fp16 大λ 1步停机 OK  kl={out.ponder_kl.item():.4f}  "
+          f"executed 步分布 w={out.ponder_weights[0, 0].tolist()}")
+
+    with torch.no_grad():
+        m.ponder_head.bias.fill_(-13.8)  # sigmoid≈1e-6 → 跑满 5 步, p 含 ~1e-6 小值
+    out = m(input_ids=ids, attention_mask=attn, labels=labels, output_ponder=True)
+    assert torch.isfinite(out.ponder_kl) and torch.isfinite(out.loss), \
+        f"[2] fp16 KL NaN/Inf (小λ): {out.ponder_kl}"
+    print(f"[2b] fp16 小λ 满步 OK  kl={out.ponder_kl.item():.4f}")
+
+    # ---------- [3] entropy / msp / drift: fp16 eval 前向 ----------
+    m.eval()
+    for sig in ("entropy", "msp", "drift"):
+        m.config.ponder_signal = sig
+        out = m(input_ids=ids, attention_mask=attn, output_ponder=True)
+        assert torch.isfinite(out.logits).all(), f"[3] {sig} logits 非有限"
+    m.config.ponder_signal = "learned"
+    print("[3] entropy/msp/drift fp16 eval 前向 OK")
+
+    # ---------- [4] gradient checkpointing 一致性 ----------
+    torch.manual_seed(2)
+    args_like = type("A", (), dict(train_mode="head+block", lr_head=1e-3,
+                                   lr_block=1e-3, lr_lora=1e-3, lora_r=8,
+                                   lora_scope="ponder"))()
+    m = PonderLlamaForCausalLM(tiny_cfg())
+    m, groups, _ = setup_trainables(m, args_like)
+    opt1 = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    m.train()
+    out1 = m(input_ids=ids, attention_mask=attn, labels=labels)
+    out1.loss.backward()
+    g1 = {n: p.grad.clone() for n, p in m.named_parameters() if p.grad is not None}
+
+    m2 = PonderLlamaForCausalLM(tiny_cfg())
+    m2.load_state_dict(m.state_dict())   # 同权重
+    m2, groups2, _ = setup_trainables(m2, args_like)
+    m2.gradient_checkpointing_enable()   # 开启重算
+    assert getattr(m2.model, "gradient_checkpointing", False), "ckpt flag 未生效"
+    m2.train()
+    out2 = m2(input_ids=ids, attention_mask=attn, labels=labels)
+    out2.loss.backward()
+    assert torch.allclose(out1.loss, out2.loss, atol=1e-4), \
+        f"[4] ckpt 开/关 loss 不一致: {out1.loss.item()} vs {out2.loss.item()}"
+    g2 = {n: p.grad for n, p in m2.named_parameters() if p.grad is not None}
+    same = all(torch.allclose(g1[n], g2[n], atol=1e-3) for n in g1 if n in g2)
+    n_match = sum(1 for n in g1 if n in g2)
+    assert same and n_match > 0, "[4] ckpt 梯度不一致"
+    print(f"[4] gradient checkpointing OK  loss={out1.loss.item():.4f}=={out2.loss.item():.4f}"
+          f"  梯度一致 {n_match} 张量")
+
+    # ---------- [5] train_one_epoch + GradScaler(disabled) 集成 ----------
+    torch.manual_seed(3)
+    m = PonderLlamaForCausalLM(tiny_cfg())
+    args_like2 = type("A", (), dict(train_mode="head", lr_head=5e-3, lr_block=1e-3,
+                                    lr_lora=1e-3, lora_r=8, lora_scope="ponder"))()
+    m, groups, _ = setup_trainables(m, args_like2)
+    opt = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: 1.0)
+    gs = train_one_epoch(m, [(ids, labels, attn)], opt, sched, "cpu",
+                         accum=1, log_every=0, scaler=None)
+    assert gs == 1
+    print("[5] train_one_epoch 集成 OK")
+
+    print("=" * 60)
+    print("ALL GPU-READINESS TESTS PASSED ✔  (T4 fp16 路径已预验证)")
+
+
+if __name__ == "__main__":
+    main()

pondernet/test_ponder.py

@@ -0,0 +1,175 @@
+# -*- coding: utf-8 -*-
+"""
+test_ponder.py — PonderLlamaForCausalLM 随机权重冒烟测试 (CPU, tiny config)
+
+验证:
+  1. prefill 前向 (思考窗口 W=1) 形状/数值合法
+  2. KV cache 长度在思考循环后严格正确 (= 真实 token 数, 不多不少)
+  3. generate() 多 token 生成 + decode 阶段逐 token 思考
+  4. 训练前向 (labels) 的 ponder loss + 反向传播 + ponder_head 梯度
+  5. 四种 halting 信号 (entropy / msp / drift / learned) 全部可跑
+  6. ponder_all_positions / commit_kv=False / max_ponder_steps=1 等开关
+"""
+import torch
+
+from ponder_llama import PonderLlamaConfig, PonderLlamaForCausalLM
+
+
+def tiny_cfg(**kw) -> PonderLlamaConfig:
+    base = dict(
+        vocab_size=256, hidden_size=64, intermediate_size=128,
+        num_hidden_layers=6, num_attention_heads=4, num_key_value_heads=2,
+        head_dim=16, max_position_embeddings=512, bos_token_id=0,
+        eos_token_id=1, pad_token_id=1,
+        ponder_start_layer=4, max_ponder_steps=5, ponder_epsilon=0.01,
+    )
+    base.update(kw)
+    return PonderLlamaConfig(**base)
+
+
+def cache_len(cache):
+    if cache is None:
+        return 0
+    return int(cache.get_seq_length())
+
+
+def main():
+    torch.manual_seed(0)
+    print("=" * 60)
+
+    # ---------- 1) prefill 前向 (entropy 信号, 默认窗口) -----------------
+    cfg = tiny_cfg()
+    model = PonderLlamaForCausalLM(cfg).eval()
+    x = torch.randint(0, 256, (1, 16))
+    with torch.no_grad():
+        out = model(x, output_ponder=True)
+    assert out.logits.shape == (1, 16, 256), out.logits.shape
+    w = out.ponder_weights
+    assert torch.allclose(w.sum(-1), torch.ones(1, 16), atol=1e-4), \
+        f"停机质量分布未归一: {w.sum(-1)}"
+    assert (w >= 0).all()
+    print(f"[1] prefill OK  logits={tuple(out.logits.shape)}  "
+          f"思考步数(最大)={out.ponder_steps.max().item()}")
+
+    # ---------- 2) KV cache 长度严格正确 --------------------------------
+    with torch.no_grad():
+        out2 = model(x, use_cache=True, output_ponder=True)
+    L = cache_len(out2.past_key_values)
+    assert L == 16, f"prefill 后 cache 长度应为 16, 实际 {L}"
+    # decode 一步
+    with torch.no_grad():
+        out3 = model(x[:, -1:], past_key_values=out2.past_key_values,
+                     use_cache=True, output_ponder=True)
+    L2 = cache_len(out3.past_key_values)
+    assert L2 == 17, f"decode 一步后 cache 长度应为 17, 实际 {L2} (思考循环泄漏/缺失 KV!)"
+    # 位置 id 连续性: 直接用 cache 里的 KV 数量核对每层一致
+    per_layer = [int(layer.get_seq_length()) for layer in out3.past_key_values.layers]
+    assert all(v == 17 for v in per_layer), f"各层 cache 长度不一致: {per_layer}"
+    print(f"[2] KV cache OK  prefill={L}  decode后={L2}  各层一致={per_layer}")
+
+    # ---------- 3) generate() 多 token 生成 ------------------------------
+    with torch.no_grad():
+        gen = model.generate(x[:, :8], max_new_tokens=6, do_sample=False)
+    assert gen.shape == (1, 14), gen.shape
+    print(f"[3] generate OK  输出长度={gen.shape[1]}  "
+          f"decode 每步思考步数={[e['steps_max'] for e in model._ponder_log if e['mode']=='decode'][:6]}")
+
+    # ---------- 4) 训练前向: ponder loss + 反向 + 梯度 -------------------
+    model_t = PonderLlamaForCausalLM(tiny_cfg(ponder_signal="learned"))
+    model_t.train()
+    xt = torch.randint(0, 256, (2, 12))
+    out_t = model_t(input_ids=xt, labels=xt, output_ponder=True)
+    assert out_t.loss is not None
+    out_t.loss.backward()
+    g_head = model_t.ponder_head.weight.grad
+    g_head_b = model_t.ponder_head.bias.grad
+    assert g_head is not None and g_head.abs().sum() > 0, "ponder_head 无梯度!"
+    assert g_head_b is not None
+    # KL 项迫使停机分布靠近几何先验: 检查 loss 数值有限
+    assert torch.isfinite(out_t.loss)
+    print(f"[4] 训练 OK  loss={out_t.loss.item():.4f}  "
+          f"ponder_head.weight.grad.abs().sum()={g_head.abs().sum().item():.6f}")
+
+    # ---------- 5) 四种 halting 信号 -------------------------------------
+    for sig in ("entropy", "msp", "drift", "learned"):
+        m = PonderLlamaForCausalLM(tiny_cfg(ponder_signal=sig)).eval()
+        with torch.no_grad():
+            o = m(x, use_cache=True, output_ponder=True)
+            g = m.generate(x[:, :6], max_new_tokens=3, do_sample=False)
+        assert o.logits.shape == (1, 16, 256) and g.shape == (1, 9)
+        assert cache_len(o.past_key_values) == 16
+        print(f"[5] signal={sig:<8} OK  思考步数(max)={o.ponder_steps.max().item()}")
+
+    # ---------- 6) 各种开关 ----------------------------------------------
+    # 6a 全序列位置思考
+    m = PonderLlamaForCausalLM(tiny_cfg(ponder_all_positions=True)).eval()
+    with torch.no_grad():
+        o = m(x, use_cache=True, output_ponder=True)
+    assert o.ponder_weights.shape[:2] == (1, 16)
+    assert cache_len(o.past_key_values) == 16
+    print(f"[6a] ponder_all_positions OK  每位置独立思考, 步数分布="
+          f"{o.ponder_steps[0].tolist()}")
+    # 6b commit_kv=False 快速模式
+    m = PonderLlamaForCausalLM(tiny_cfg(commit_kv=False)).eval()
+    with torch.no_grad():
+        o = m(x, use_cache=True)
+        g = m.generate(x[:, :6], max_new_tokens=3, do_sample=False)
+    assert cache_len(o.past_key_values) == 16 and g.shape == (1, 9)
+    print("[6b] commit_kv=False OK")
+    # 6c max_ponder_steps=1 (退化为普通前向)
+    m1 = PonderLlamaForCausalLM(tiny_cfg(max_ponder_steps=1)).eval()
+    mb = PonderLlamaForCausalLM(tiny_cfg()).eval()  # 对照: 与 base 完全一致?
+    with torch.no_grad():
+        o1 = m1(x)
+    print("[6c] max_ponder_steps=1 OK")
+    # 6d batch>1 + padding mask
+    xb = torch.randint(0, 256, (2, 10))
+    mask = torch.ones_like(xb)
+    mask[1, 6:] = 0  # 右 padding
+    m = PonderLlamaForCausalLM(tiny_cfg()).eval()
+    with torch.no_grad():
+        ob = m(xb, attention_mask=mask, use_cache=True)
+    assert ob.logits.shape == (2, 10, 256)
+    assert cache_len(ob.past_key_values) == 10
+    print("[6d] batch+padding mask OK")
+
+    # ---------- 7) 概率混合 vs 单步: 数值一致性抽查 ----------------------
+    # executed==1 时 mix 应精确等于单步输出 (commit 自动跳过路径)
+    m = PonderLlamaForCausalLM(tiny_cfg(max_ponder_steps=3, ponder_epsilon=0.01)).eval()
+    with torch.no_grad():
+        o = m(x[:, :10], use_cache=True, output_ponder=True)
+    w0 = o.ponder_weights[0, 0]
+    # 单位置思考: 若第 1 步就停 (w[0]≈1), 逻辑上 cache 长度仍须正确
+    assert torch.isclose(w0.sum(), torch.tensor(1.0), atol=1e-4)
+    assert cache_len(o.past_key_values) == 10
+    print(f"[7] 单位置思考 OK  w={w0.tolist()}")
+
+    # ---------- 8) eager attention 后端 (手工加性 mask 兼容性) -----------
+    m = PonderLlamaForCausalLM(tiny_cfg()).eval()
+    m.config._attn_implementation = "eager"
+    with torch.no_grad():
+        o = m(x, use_cache=True, output_ponder=True)
+        g = m.generate(x[:, :6], max_new_tokens=3, do_sample=False)
+    assert o.logits.shape == (1, 16, 256) and g.shape == (1, 9)
+    assert cache_len(o.past_key_values) == 16
+    print("[8] eager attention OK")
+
+    # ---------- 9) 左 padding + generate (chat 默认场景) ------------------
+    m = PonderLlamaForCausalLM(tiny_cfg()).eval()
+    xl = torch.randint(0, 256, (2, 10))
+    pad_mask = torch.ones_like(xl)
+    pad_mask[1, :4] = 0                     # 第 2 条左 pad 4 个
+    with torch.no_grad():
+        g = m.generate(xl, attention_mask=pad_mask, max_new_tokens=4, do_sample=False)
+    assert g.shape == (2, 14), g.shape
+    steps = [e for e in m._ponder_log if e["mode"] == "decode"]
+    assert 1 <= len(steps) <= 4, f"decode 步数异常: {len(steps)}"  # 随机权重可能提前生成 EOS
+    print(f"[9] 左 padding generate OK  实际 decode {len(steps)} 步, "
+          f"每步思考步数={[e['steps_max'] for e in steps]}")
+
+    print("=" * 60)
+    print("ALL TESTS PASSED ✔")
+
+
+if __name__ == "__main__":
+    main()

pondernet/test_train.py

@@ -0,0 +1,173 @@
+# -*- coding: utf-8 -*-
+"""test_train.py — train_ponder_head.py 训练管线的 tiny 端到端验证 (CPU)
+
+验证:
+  1. 演示数据生成 + chat 数据管线 (assistant 区间才计 loss)
+  2. head 模式: 仅 ponder_head 可训, loss 有限, 主干梯度为空, 步数分布变化
+  3. head+block 模式: CE 显著下降 (思考块学会被重复执行)
+  4. head+lora 模式: peft 包装 + 训练 + ponder_head/adapter 双产物保存重载
+"""
+import json
+import os
+import tempfile
+
+import torch
+from torch.utils.data import DataLoader
+
+from ponder_llama import PonderLlamaConfig, PonderLlamaForCausalLM
+from train_ponder_head import (DEMO_DATA, PonderDataset, collate, load_jsonl,
+                               make_demo_data, setup_trainables, train_one_epoch)
+
+TOK_DIR = "/home/z/my-project/models/MiniCPM5-2B-cpu"
+
+
+def tiny_cfg():
+    return PonderLlamaConfig(
+        vocab_size=256, hidden_size=64, intermediate_size=128,
+        num_hidden_layers=6, num_attention_heads=4, num_key_value_heads=2,
+        head_dim=16, max_position_embeddings=512, bos_token_id=0,
+        eos_token_id=1, pad_token_id=1,
+        ponder_start_layer=4, max_ponder_steps=5, ponder_epsilon=0.01,
+        ponder_signal="learned", ponder_all_positions=True,
+        ponder_prior_p=0.4, ponder_loss_beta=0.05)
+
+
+def make_args(**kw):
+    """构造 argparse.Namespace 替身"""
+    defaults = dict(train_mode="head", lr_head=3e-3, lr_lora=1e-3, lr_block=1e-3,
+                    lora_r=8, lora_scope="ponder", max_steps=5, epsilon=0.01,
+                    prior_p=0.4, beta=0.05, head_bias_init=-1.0, start_layer=None,
+                    grad_ckpt=False)
+    defaults.update(kw)
+
+    class A:  # 轻量 namespace
+        def __init__(self, d):
+            self.__dict__.update(d)
+    return A(defaults)
+
+
+def synthetic_batches(vocab=256, n=8, seqlen=24, batch=4, seed=0):
+    """固定合成任务: 每条序列是一个可记忆的重复模式 (供 CE 下降验证)"""
+    g = torch.Generator().manual_seed(seed)
+    seqs = torch.randint(3, vocab, (n, seqlen), generator=g)
+    data = []
+    for i in range(n):
+        s = seqs[i % n]
+        ids = torch.cat([s, s.flip(0)])[:seqlen]     # 结构化模式, 可学习
+        labels = ids.clone()
+        data.append((ids.unsqueeze(0), labels.unsqueeze(0),
+                     torch.ones(1, seqlen, dtype=torch.long)))
+    return data
+
+
+def main():
+    torch.manual_seed(0)
+    dev = "cpu"
+
+    # ---------- 1) 演示数据 + chat 数据管线 ------------------------------
+    with tempfile.TemporaryDirectory() as td:
+        demo_path = os.path.join(td, "demo.jsonl")
+        make_demo_data(demo_path)
+        rows = load_jsonl(demo_path)
+        assert len(rows) == len(DEMO_DATA)
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(TOK_DIR, trust_remote_code=True)
+    ds = PonderDataset(rows[:3], tok, max_len=256)
+    b_ids, b_labels, b_attn = collate([ds[0], ds[1]], pad_id=1)
+    # assistant 区间应有监督, user/system 区间无监督
+    supervised = (b_labels != -100).float().mean().item()
+    assert 0.0 < supervised < 0.6, f"监督比例异常: {supervised}"
+    assert (b_labels == -100).sum() > 0 and (b_labels != -100).sum() > 0
+    print(f"[1] 数据管线 OK  batch={tuple(b_ids.shape)}  监督比例={supervised:.2%}")
+
+    # ---------- 2) head 模式: 仅 ponder_head 可训 -------------------------
+    model = PonderLlamaForCausalLM(tiny_cfg()).to(dev)
+    args = make_args(train_mode="head", lr_head=5e-3)
+    model, groups, note = setup_trainables(model, args)
+    trainable = {n for n, p in model.named_parameters() if p.requires_grad}
+    assert all(n.startswith("ponder_head") for n in trainable), trainable
+    n_backbone_grad = sum(1 for n, p in model.named_parameters()
+                          if p.requires_grad and not n.startswith("ponder_head"))
+    assert n_backbone_grad == 0
+    loader = [(torch.cat([x[0] for x in synthetic_batches()[:4]]),
+               torch.cat([x[1] for x in synthetic_batches()[:4]]),
+               torch.cat([x[2] for x in synthetic_batches()[:4]]))]
+    opt = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: 1.0)
+    gs = train_one_epoch(model, loader, opt, sched, dev, accum=1, log_every=0)
+    assert gs == 1
+    # train_one_epoch 内部 step 后会清梯度; 单独前向+反传验证梯度路径
+    ids0, labels0, attn0 = loader[0]
+    model.train()
+    out = model(input_ids=ids0, attention_mask=attn0, labels=labels0)
+    out.loss.backward()
+    grads = [p.grad for p in model.ponder_head.parameters()]
+    assert grads[0] is not None and grads[0].abs().sum() > 0, "ponder_head 无梯度"
+    print(f"[2] head 模式 OK  可训参数={[n for n in trainable]}  "
+          f"head 梯度范数={grads[0].abs().mean().item():.2e}")
+    print(f"    冻结验证 OK  主干可训参数量=0")
+
+    # ---------- 3) head+block 模式: CE 显著下降 --------------------------
+    torch.manual_seed(1)
+    model = PonderLlamaForCausalLM(tiny_cfg()).to(dev)
+    args = make_args(train_mode="head+block", lr_block=3e-3, lr_head=5e-3)
+    model, groups, note = setup_trainables(model, args)
+    n_train = sum(p.numel() for p in model.parameters() if p.requires_grad)
+    assert n_train > 50_000, f"思考块应可训 (实际 {n_train})"
+    loader = [(torch.cat([x[0] for x in synthetic_batches()]),
+               torch.cat([x[1] for x in synthetic_batches()]),
+               torch.cat([x[2] for x in synthetic_batches()]))]
+    opt = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: 1.0)
+    model.eval = model.eval  # noop, 保持 train() 切换由 train_one_epoch 控制
+    first_ce = None
+    for i in range(100):
+        gs = train_one_epoch(model, loader, opt, sched, dev, accum=1, log_every=0)
+    # 重新前向测 CE
+    model.train()
+    ids, labels, attn = loader[0]
+    with torch.no_grad():
+        out = model(input_ids=ids, attention_mask=attn, labels=labels)
+    ce_final = out.loss.item() - 0.05 * out.ponder_kl.item()
+    print(f"[3] head+block OK  训后 CE={ce_final:.4f} (随机基线≈ln(256)={torch.log(torch.tensor(256.)).item():.2f})")
+    assert ce_final < 4.5, f"CE 未有效下降: {ce_final}"
+
+    # ---------- 4) head+lora 模式: 包装/训练/保存重载 ---------------------
+    torch.manual_seed(2)
+    model = PonderLlamaForCausalLM(tiny_cfg()).to(dev)
+    args = make_args(train_mode="head+lora", lr_lora=1e-2, lr_head=5e-3, lora_r=8)
+    model, groups, note = setup_trainables(model, args)
+    names = [n for n, p in model.named_parameters() if p.requires_grad]
+    assert any("lora_" in n for n in names), "LoRA 参数未注入"
+    assert any(n.startswith("ponder_head") or "ponder_head" in n for n in names), \
+        "ponder_head 应可训"
+    opt = torch.optim.AdamW([{"params": g["params"], "lr": g["lr"]} for g in groups])
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lambda s: 1.0)
+    gs = train_one_epoch(model, loader, opt, sched, dev, accum=1, log_every=0)
+    with tempfile.TemporaryDirectory() as td:
+        from train_ponder_head import save_model
+        save_model(model, args, td)
+        files = set(os.listdir(td))
+        assert "ponder_head.safetensors" in files, files
+        assert "adapter_model.safetensors" in files, files
+        # 重载 ponder_head (对 Linear 模块加载需去掉模块名前缀)
+        from safetensors.torch import load_file
+        head = load_file(os.path.join(td, "ponder_head.safetensors"))
+        core = model.get_base_model()
+        core.ponder_head.load_state_dict(
+            {k.replace("ponder_head.", ""): v for k, v in head.items()})
+    print(f"[4] head+lora OK  可训: {len(names)} 个张量 "
+          f"(lora {sum(1 for n in names if 'lora_' in n)} + head)  产物: {sorted(files)}")
+
+    # ---------- 5) ponder_kl 字段 ----------------------------------------
+    with torch.no_grad():
+        out = model(input_ids=ids, attention_mask=attn, labels=labels)
+    assert out.ponder_kl is not None and torch.isfinite(out.ponder_kl)
+    print(f"[5] ponder_kl 字段 OK  KL={out.ponder_kl.item():.4f}")
+
+    print("=" * 60)
+    print("ALL TRAIN TESTS PASSED ✔")
+
+
+if __name__ == "__main__":
+    main()

pondernet/train_ponder_head.py

@@ -0,0 +1,667 @@
+# -*- coding: utf-8 -*-
+"""
+train_ponder_head.py — 微调 PonderLlamaForCausalLM 的 halting 头 (真 PonderNet 训练)
+
+三种训练模式 (--train-mode):
+  head       只训 ponder_head (2049 参数), 主干全部冻结。最便宜, 单卡 8GB 也能跑;
+             但"多想几步"带来的增益有限 (思考块没学过被重复执行)。
+  head+lora  ponder_head + LoRA (--lora-scope ponder 只注入思考块 / all 注入全部层)。
+             推荐: 让思考块学会"被重复执行", 完整 PonderNet 语义, 显存友好的代价。
+  head+block ponder_head + 思考块全参微调。效果上限最高, 显存/算力开销最大。
+
+数据格式 (jsonl, 每行一条):
+  {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
+  或 {"text": "一段完整文本"}          (用 --text-field text)
+可用 --make-demo-data demo.jsonl 生成 16 条中/英混合演示数据 (简单问答 + 推理题,
+让 halting 头有机会学到"难题多想、简单题少想")。
+
+日志中的关键指标是 avg_ponder_steps: 训练初期接近 max_steps, 随 CE+KL 联合作用,
+简单题的步数应逐步下降 —— 这就是模型在学"该想几步"。
+
+示例:
+  python train_ponder_head.py --model /path/to/MiniCPM5-2B-cpu \
+      --train-mode head+lora --lora-scope ponder --data data.jsonl \
+      --epochs 2 --output out/ponder-lora
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import os
+import random
+import time
+
+import torch
+import torch.nn.functional as F
+from torch.utils.data import DataLoader, Dataset
+
+from transformers import AutoTokenizer, get_cosine_schedule_with_warmup
+
+from ponder_llama import PonderLlamaConfig, PonderLlamaForCausalLM, LlamaConfig
+
+
+# ========================================================================== #
+# 数据
+# ========================================================================== #
+DEMO_DATA = [
+    {"messages": [{"role": "user", "content": "你好"}, {"role": "assistant", "content": "你好！很高兴见到你。"}]},
+    {"messages": [{"role": "user", "content": "1+1等于几？"}, {"role": "assistant", "content": "1+1=2。"}]},
+    {"messages": [{"role": "user", "content": "今天星期几"}, {"role": "assistant", "content": "抱歉，我无法获取实时日期。"}]},
+    {"messages": [{"role": "user", "content": "把'苹果'翻译成英文"}, {"role": "assistant", "content": "苹果的英文是 apple。"}]},
+    {"messages": [{"role": "user", "content": "天空是什么颜色"}, {"role": "assistant", "content": "晴天时天空通常是蓝色的。"}]},
+    {"messages": [{"role": "user", "content": "水在多少度结冰？"}, {"role": "assistant", "content": "标准大气压下，水在 0 摄氏度结冰。"}]},
+    {"messages": [{"role": "user", "content": "9.11 和 9.9 哪个大？"}, {"role": "assistant", "content": "9.9 更大。比较小数：9.11 = 9 + 0.11，9.9 = 9 + 0.90，因为 0.90 > 0.11，所以 9.9 > 9.11。"}]},
+    {"messages": [{"role": "user", "content": "一个水池甲管8小时注满，乙管12小时注满，两管同开几小时注满？"}, {"role": "assistant", "content": "设总量为1。甲每小时注 1/8，乙每小时注 1/12，合计 1/8+1/12 = 3/24+2/24 = 5/24。注满需要 24/5 = 4.8 小时。"}]},
+    {"messages": [{"role": "user", "content": "鸡兔同笼，头36，脚100，各几只？"}, {"role": "assistant", "content": "设鸡 x 只，兔 36-x 只。脚数 2x + 4(36-x) = 100，得 144-2x = 100，x = 22。鸡 22 只，兔 14 只。"}]},
+    {"messages": [{"role": "user", "content": "证明 sqrt(2) 是无理数"}, {"role": "assistant", "content": "反证法：若 sqrt(2)=p/q（最简分数），则 p^2=2q^2，故 p 为偶数，设 p=2k，则 4k^2=2q^2，q^2=2k^2，q 也为偶数，与 p/q 最简矛盾。故 sqrt(2) 无理。"}]},
+    {"messages": [{"role": "user", "content": "写一个Python函数判断质数"}, {"role": "assistant", "content": "def is_prime(n):\n    if n < 2: return False\n    for i in range(2, int(n**0.5)+1):\n        if n % i == 0: return False\n    return True"}]},
+    {"messages": [{"role": "user", "content": "如果所有的A都是B，所有的B都是C，那么所有的A都是C吗？"}, {"role": "assistant", "content": "是的。这是三段论的传递性：A⊆B 且 B⊆C，则 A⊆C。"}]},
+    {"messages": [{"role": "user", "content": "给我讲个笑话"}, {"role": "assistant", "content": "程序员最讨厌的两件事：别人不写注释，和别人让自己写注释。"}]},
+    {"messages": [{"role": "user", "content": "一张纸对折10次后有多少层？"}, {"role": "assistant", "content": "每对折一次层数翻倍，对折 10 次后是 2^10 = 1024 层。"}]},
+    {"messages": [{"role": "user", "content": "用'但是'造句"}, {"role": "assistant", "content": "他很想去看电影，但是作业还没写完。"}]},
+    {"messages": [{"role": "user", "content": "5个人互相握手一次，共握手几次？"}, {"role": "assistant", "content": "这是组合问题：C(5,2) = 5×4/2 = 10 次。"}]},
+]
+
+
+def make_demo_data(path: str):
+    with open(path, "w", encoding="utf-8") as f:
+        for item in DEMO_DATA:
+            f.write(json.dumps(item, ensure_ascii=False) + "\n")
+    print(f"演示数据已写入 {path} ({len(DEMO_DATA)} 条)")
+
+
+# ========================================================================== #
+# 难度分级数据生成 (easy / medium / hard)
+# 设计目的: PonderNet 的核心卖点是"难题多想、简单题少想"。只有当数据存在清晰
+# 的难度梯度时, halting 头才有信号可学 —— 训练后各难度桶的平均思考步数应当分化。
+# 全部题目反向构造 (先选解再出题), 保证答案正确; "difficulty" 字段供 eval 分层统计。
+# ========================================================================== #
+EASY_GREET = [
+    ("你好", "你好！很高兴见到你，有什么可以帮你的吗？"),
+    ("你好呀", "你好呀！今天过得怎么样？"),
+    ("早上好", "早上好！祝你今天心情愉快。"),
+    ("晚安", "晚安，祝你做个好梦。"),
+    ("谢谢", "不客气，能帮到你我很开心。"),
+    ("你是谁", "我是一个 AI 助手，可以回答问题、陪你聊天。"),
+    ("你会什么", "我可以聊天、答题、算数，也可以帮你翻译。"),
+    ("再见", "再见！欢迎随时来找我聊天。"),
+    ("你叫什么名字", "你可以叫我小助手，很高兴认识你。"),
+    ("今天天气不错", "是啊，这样的天气很适合出门走走。"),
+    ("讲个笑话", "程序员最讨厌两件事：别人不写注释，和让自己写注释。"),
+    ("你吃饭了吗", "我是 AI 不用吃饭，不过谢谢你关心！"),
+    ("中午好", "中午好！记得吃午饭休息一下哦。"),
+    ("在吗", "在的，请说。"),
+    ("谢谢你", "不用谢，这是我应该做的。"),
+    ("你会说英文吗", "会的，你可以用英文和我聊天。"),
+    ("新年快乐", "新年快乐！祝万事如意。"),
+    ("生日快乐", "谢谢！也祝你天天开心。"),
+    ("周末愉快", "周末愉快！好好放松一下吧。"),
+    ("下午好", "下午好！喝杯茶休息一下吧。"),
+]
+COMMON_SENSE = [
+    ("天空是什么颜色", "晴天时天空通常是蓝色的。"),
+    ("一年有几个月", "一年有 12 个月。"),
+    ("一个星期有几天", "一个星期有 7 天。"),
+    ("冰遇到热会怎么样", "冰遇到热会融化成水。"),
+    ("太阳从哪边升起", "太阳从东边升起，西边落下。"),
+    ("中国的首都是哪里", "中国的首都是北京。"),
+    ("一年有哪四个季节", "一年有春、夏、秋、冬四个季节。"),
+    ("彩虹有几种颜色", "彩虹有红橙黄绿蓝靛紫 7 种颜色。"),
+    ("水在多少度结冰", "标准大气压下，水在 0 摄氏度结冰。"),
+    ("水在多少度沸腾", "标准大气压下，水在 100 摄氏度沸腾。"),
+    ("世界上最大的海洋是哪个", "世界上最大的海洋是太平洋。"),
+    ("熊猫最爱吃什么", "熊猫最爱吃竹子。"),
+    ("蜜蜂酿什么", "蜜蜂采花蜜酿成蜂蜜。"),
+    ("植物生长需要什么", "植物生长一般需要阳光、水分和空气。"),
+    ("十二生肖排第一的是什么", "十二生肖排第一的是鼠。"),
+    ("中国最长的河流是哪条", "中国最长的河流是长江。"),
+    ("世界上最高的山峰是哪座", "世界上最高的山峰是珠穆朗玛峰。"),
+    ("人的正常体温大约是多少", "人的正常体温大约是 37 摄氏度。"),
+    ("一天有多少个小时", "一天有 24 个小时。"),
+    ("一公里等于多少米", "一公里等于 1000 米。"),
+]
+TRANSLATE = [
+    ("苹果", "apple"), ("书", "book"), ("猫", "cat"), ("狗", "dog"),
+    ("太阳", "sun"), ("月亮", "moon"), ("水", "water"), ("火", "fire"),
+    ("山", "mountain"), ("河", "river"), ("树", "tree"), ("花", "flower"),
+    ("鸟", "bird"), ("鱼", "fish"), ("学校", "school"), ("老师", "teacher"),
+    ("学生", "student"), ("医生", "doctor"), ("医院", "hospital"), ("汽车", "car"),
+    ("火车", "train"), ("飞机", "airplane"), ("手机", "phone"), ("电脑", "computer"),
+    ("蓝色", "blue"), ("红色", "red"), ("绿色", "green"), ("白色", "white"),
+    ("星期一", "Monday"), ("星期天", "Sunday"), ("早上", "morning"), ("晚上", "evening"),
+    ("米饭", "rice"), ("面包", "bread"), ("鸡蛋", "egg"), ("牛奶", "milk"),
+    ("茶", "tea"), ("咖啡", "coffee"), ("朋友", "friend"), ("家庭", "family"),
+]
+ANTONYMS = [
+    ("大", "小"), ("多", "少"), ("高", "矮"), ("长", "短"), ("快", "慢"),
+    ("上", "下"), ("左", "右"), ("前", "后"), ("开", "关"), ("好", "坏"),
+    ("新", "旧"), ("冷", "热"), ("黑", "白"), ("买", "卖"), ("来", "去"),
+    ("远", "近"), ("深", "浅"), ("宽", "窄"), ("厚", "薄"), ("轻", "重"),
+    ("满", "空"), ("强", "弱"), ("甜", "苦"), ("明亮", "黑暗"), ("开始", "结束"),
+    ("���功", "失败"), ("勇敢", "胆怯"), ("勤劳", "懒惰"), ("干净", "肮脏"), ("整齐", "杂乱"),
+]
+# 工程问题预解: (a, b, t) 满足 1/a + 1/b = 1/t, 即 (a-t)(b-t) = t^2
+WORK_PROBLEMS = [(3, 6, 2), (4, 4, 2), (4, 12, 3), (6, 6, 3), (5, 20, 4),
+                 (6, 12, 4), (8, 8, 4), (6, 30, 5), (10, 10, 5), (7, 42, 6),
+                 (8, 24, 6), (9, 18, 6), (10, 15, 6), (12, 12, 6)]
+
+
+def _gen_easy(rng):
+    kind = rng.randrange(4)
+    if kind == 0:
+        q, a = rng.choice(EASY_GREET)
+    elif kind == 1:
+        q, a = rng.choice(COMMON_SENSE)
+    elif kind == 2:
+        zh, en = rng.choice(TRANSLATE)
+        if rng.random() < 0.5:
+            q, a = f"把'{zh}'翻译成英文", f"{zh} 的英文是 {en}。"
+        else:
+            q, a = f"把'{en}'翻译成中文", f"{en} 的中文是 {zh}。"
+    else:
+        op = rng.choice(["+", "-"])
+        x, y = rng.randint(1, 9), rng.randint(1, 9)
+        if op == "-":
+            x, y = max(x, y), min(x, y)
+        res = x + y if op == "+" else x - y
+        q, a = f"{x} {op} {y} 等于几？", f"{x} {op} {y} = {res}。"
+    return q, a
+
+
+def _gen_medium(rng):
+    kind = rng.randrange(5)
+    if kind == 0:  # 两位数乘一位数 / 三位数加减
+        if rng.random() < 0.5:
+            a, b = rng.randint(10, 99), rng.randint(2, 9)
+            q, a_ = f"{a} 乘 {b} 等于多少？", f"{a} × {b} = {a * b}。"
+        else:
+            a, b = rng.randint(100, 999), rng.randint(100, 999)
+            if rng.random() < 0.5:
+                q, a_ = f"{a} 加 {b} 等于多少？", f"{a} + {b} = {a + b}。"
+            else:
+                hi, lo = max(a, b), min(a, b)
+                q, a_ = f"{hi} 减 {lo} 等于多少？", f"{hi} - {lo} = {hi - lo}。"
+    elif kind == 1:  # 单位换算
+        table = [("米", "厘米", 100), ("米", "毫米", 1000), ("千克", "克", 1000),
+                 ("元", "角", 10), ("小时", "分钟", 60), ("分", "秒", 60)]
+        big, small, rate = rng.choice(table)
+        v = rng.randint(2, 99)
+        if rng.random() < 0.5:
+            q, a_ = f"{v} {big} 等于多少 {small}？", f"{v} {big} = {v * rate} {small}。"
+        else:
+            q, a_ = (f"{v * rate} {small} 等于多少 {big}？",
+                     f"{v * rate} {small} = {v} {big}。")
+    elif kind == 2:  # 单价×数量
+        p, n = rng.randint(2, 20), rng.randint(2, 12)
+        q, a_ = (f"一支笔 {p} 元，买 {n} 支一共需要多少钱？",
+                 f"单价 {p} 元，数量 {n} 支，一共 {p} × {n} = {p * n} 元。")
+    elif kind == 3:  # 速度×时间
+        v, t = rng.choice([20, 30, 40, 50, 60, 70, 80]), rng.randint(2, 9)
+        q, a_ = (f"一辆汽车每小时行驶 {v} 千米，行驶 {t} 小时，一共行驶多少千米？",
+                 f"速度 {v} 千米/小时 × 时间 {t} 小时 = {v * t} 千米。")
+    else:  # 反义词
+        w, ant = rng.choice(ANTONYMS)
+        q, a_ = f"'{w}'的反义词是什么？", f"'{w}'的反义词是'{ant}'。"
+    return q, a_
+
+
+def _gen_hard(rng):
+    kind = rng.randrange(6)
+    if kind == 0:  # 三位数×两位数, 答案带分解步骤
+        a, b = rng.randint(100, 999), rng.randint(10, 99)
+        tens, ones = b // 10 * 10, b % 10
+        q = f"{a} 乘 {b} 等于多少？"
+        a_ = (f"{a} × {b} = {a * b}。把 {b} 拆成 {tens} + {ones}："
+              f"{a} × {tens} = {a * tens}，{a} × {ones} = {a * ones}，"
+              f"相加得 {a * tens + a * ones} = {a * b}。")
+    elif kind == 1:  # 鸡兔同笼 (反向构造保证整数解)
+        c, r = rng.randint(2, 20), rng.randint(2, 20)
+        heads, feet = c + r, 2 * c + 4 * r
+        q = f"鸡兔同笼，从上面数有 {heads} 个头，从下面数有 {feet} 只脚，鸡和兔各有几只？"
+        a_ = (f"设鸡 x 只，兔 {heads} - x 只。列方程 2x + 4({heads} - x) = {feet}，"
+              f"化简得 {4 * heads} - 2x = {feet}，x = {c}。"
+              f"所以鸡 {c} 只，兔 {r} 只。")
+    elif kind == 2:  # 相遇问题
+        v1, v2 = rng.choice([20, 30, 40, 50, 60]), rng.choice([20, 30, 40, 50, 60])
+        t = rng.randint(2, 6)
+        d = (v1 + v2) * t
+        q = (f"甲、乙两车分别从相距 {d} 千米的两地同时出发，相向而行。"
+             f"甲车每小时行 {v1} 千米，乙车每小时行 {v2} 千米，几小时后两车相遇？")
+        a_ = (f"两车速度和为 {v1} + {v2} = {v1 + v2} 千米/小时。"
+              f"相遇时间 = 距离 ÷ 速度和 = {d} ÷ {v1 + v2} = {t} 小时。")
+    elif kind == 3:  # 工程问题 (预解表保证答案整洁)
+        a, b, t = rng.choice(WORK_PROBLEMS)
+        q = f"一个水池，甲管单独注满需要 {a} 小时，乙管单独注满需要 {b} 小时，两管同时开需要几小时注满？"
+        a_ = (f"设总量为 1。甲管每小时注 1/{a}，乙管每小时注 1/{b}，"
+              f"两管同开每小时注 1/{a} + 1/{b} = {a + b}/{a * b}。"
+              f"注满需要 {a * b} ÷ {a + b} = {t} 小时。")
+    elif kind == 4:  # 数列推理 (等差/等比/二阶)
+        sub = rng.randrange(3)
+        if sub == 0:
+            a1, d = rng.randint(2, 20), rng.randint(2, 12)
+            seq = [a1 + i * d for i in range(5)]
+            q, a_ = (f"找规律：{', '.join(map(str, seq))}，下一个数是多少？",
+                     f"相邻两项的差恒为 {d}，下一个数是 {seq[-1]} + {d} = {seq[-1] + d}。")
+        elif sub == 1:
+            a1, r = rng.randint(1, 4), rng.choice([2, 3])
+            seq, v = [], a1
+            for _ in range(5):
+                seq.append(v)
+                v *= r
+            q, a_ = (f"找规律：{', '.join(map(str, seq))}，下一个数是多少？",
+                     f"相邻两项的比为 {r}，下一个数是 {seq[-1]} × {r} = {seq[-1] * r}。")
+        else:
+            a1, d0, e = rng.randint(1, 9), rng.randint(1, 5), rng.randint(1, 4)
+            seq, v, dd = [], a1, d0
+            for _ in range(5):
+                seq.append(v)
+                v += dd
+                dd += e
+            q, a_ = (f"找规律：{', '.join(map(str, seq))}，下一个数是多少？",
+                     f"相邻两项的差依次是 {', '.join(str(d0 + i * e) for i in range(4))}，"
+                     f"差递增 {e}。下一个差是 {dd}，下一个数是 {seq[-1]} + {dd} = {seq[-1] + dd}。")
+    else:  # 除法余数 / 浓度
+        if rng.random() < 0.5:
+            divisor = rng.randint(6, 12)
+            r_ = rng.randint(1, divisor - 1)
+            quot = rng.randint(10, 99)
+            n = divisor * quot + r_
+            q = f"{n} 除以 {divisor}，商和余数分别是多少？"
+            a_ = f"{n} ÷ {divisor} = {quot} 余 {r_}，即商是 {quot}，余数是 {r_}。"
+        else:
+            c = rng.choice([10, 20, 25, 50])
+            m = rng.choice([200, 400, 500, 800, 1000])
+            sugar = m * c // 100
+            q = f"{m} 克糖水中含糖 {sugar} 克，这种糖水的浓度是多少？"
+            a_ = f"浓度 = 糖 ÷ 糖水 = {sugar} ÷ {m} = {c}%。"
+    return q, a_
+
+
+def make_mix_data(train_path: str, n_easy=220, n_medium=200, n_hard=180,
+                  n_eval_per_bucket=30, seed=7):
+    """生成难度分级数据集: train_path (训练) + train_path 去掉 .jsonl 加 .eval.jsonl (评估)。
+    每行: {"messages": [...], "difficulty": "easy|medium|hard"}"""
+    rng = random.Random(seed)
+    eval_path = train_path[:-len(".jsonl")] + ".eval.jsonl" if train_path.endswith(".jsonl") \
+        else train_path + ".eval.jsonl"
+    gens = {"easy": _gen_easy, "medium": _gen_medium, "hard": _gen_hard}
+
+    def fill(bucket, n, used, salt):
+        rows, tries = [], 0
+        r2 = random.Random(seed + salt)
+        while len(rows) < n and tries < n * 50:
+            tries += 1
+            q, a = gens[bucket](r2)
+            if q in used:
+                continue
+            used.add(q)
+            rows.append({"messages": [{"role": "user", "content": q},
+                                      {"role": "assistant", "content": a}],
+                         "difficulty": bucket})
+        return rows
+
+    used = set()
+    buckets = ["easy", "medium", "hard"]
+    train_rows = {b: fill(b, n, used, i)
+                  for i, (b, n) in enumerate(zip(buckets, [n_easy, n_medium, n_hard]))}
+    eval_rows = {b: fill(b, n_eval_per_bucket, used, 100) for b in buckets}
+
+    for path, rows in [(train_path, train_rows), (eval_path, eval_rows)]:
+        with open(path, "w", encoding="utf-8") as f:
+            for b in ("easy", "medium", "hard"):
+                for r in rows[b]:
+                    f.write(json.dumps(r, ensure_ascii=False) + "\n")
+        counts = {b: len(rows[b]) for b in rows}
+        print(f"已写入 {path}  {counts}")
+
+
+def load_jsonl(path: str) -> list[dict]:
+    rows = []
+    with open(path, encoding="utf-8") as f:
+        for line in f:
+            line = line.strip()
+            if line:
+                rows.append(json.loads(line))
+    return rows
+
+
+def _apply_chat_ids(tok, msgs, add_generation_prompt=False) -> list[int]:
+    """apply_chat_template 跨版本归一化: 统一返回 list[int]
+    (transformers 5.x 返回 BatchEncoding, 旧版返回 list[int], fast 分支返回 Encoding)"""
+    out = tok.apply_chat_template(msgs, tokenize=True,
+                                  add_generation_prompt=add_generation_prompt)
+    if hasattr(out, "ids"):                       # tokenizers.Encoding
+        return list(out.ids)
+    if hasattr(out, "input_ids"):                 # BatchEncoding
+        out = out["input_ids"]
+    if isinstance(out, torch.Tensor):
+        out = out.tolist()
+    if out and isinstance(out[0], (list, tuple)):  # 嵌套 -> 单条
+        out = out[0]
+    return [int(t) for t in out]
+
+
+class PonderDataset(Dataset):
+    """jsonl -> token ids。chat 格式走 chat_template 并只对回答部分计 loss;
+    纯文本格式整句计 loss。右侧 padding, labels 在 pad 处置 -100。"""
+
+    def __init__(self, rows, tokenizer, max_len=512, text_field=None):
+        tok = tokenizer
+        self.tok = tokenizer
+        self.max_len = max_len
+        self.examples = []
+        pad_id = tokenizer.pad_token_id or 0
+        for r in rows:
+            if text_field and text_field in r:
+                enc = tokenizer(r[text_field], truncation=True, max_length=max_len)
+                ids, label_mask = enc["input_ids"], [1] * len(enc["input_ids"])
+            else:
+                msgs = r["messages"]
+                # 逐轮拼接: assistant 回答的 token 区间计 loss, 其余位置置 -100
+                ids: list[int] = []
+                label_mask: list[int] = []
+                for i, m in enumerate(msgs):
+                    cur = _apply_chat_ids(tok, msgs[: i + 1])
+                    new = cur[len(ids):] if cur[: len(ids)] == ids else cur
+                    label_mask += ([1] if m["role"] == "assistant" else [0]) * len(new)
+                    ids = cur
+            ids = ids[:max_len]
+            label_mask = label_mask[: len(ids)]
+            labels = [t if mk == 1 else -100 for t, mk in zip(ids, label_mask)]
+            self.examples.append({"input_ids": ids, "labels": labels})
+
+    def __len__(self):
+        return len(self.examples)
+
+    def __getitem__(self, i):
+        return self.examples[i]
+
+
+def collate(batch, pad_id=1):
+    maxlen = max(len(b["input_ids"]) for b in batch)
+    input_ids, labels, attn = [], [], []
+    for b in batch:
+        n = len(b["input_ids"])
+        pad = maxlen - n
+        input_ids.append(b["input_ids"] + [pad_id] * pad)
+        labels.append(b["labels"] + [-100] * pad)
+        attn.append([1] * n + [0] * pad)
+    return (torch.tensor(input_ids), torch.tensor(labels), torch.tensor(attn))
+
+
+# ========================================================================== #
+# 模型 / 参数分组
+# ========================================================================== #
+def resolve_dtype(arg, device):
+    """--dtype 解析。auto: cpu→fp32; cuda→原生 bf16 (sm80+) 否则 fp16。
+    注意: 不能用 torch.cuda.is_bf16_supported() 判断 T4 —— 新版 torch 对 sm75
+    返回 True 但那是模拟 (emulated) bf16, 没有硬件 tensor core, 训练极慢。
+    T4 (sm75) → fp16 + GradScaler; 30/40/50 系 (sm80+) → 原生 bf16。"""
+    if arg == "float32":
+        return torch.float32
+    if arg == "float16":
+        return torch.float16
+    if arg == "bfloat16":
+        return torch.bfloat16
+    if device == "cpu":
+        return torch.float32
+    major, _ = torch.cuda.get_device_capability(device)
+    return torch.bfloat16 if major >= 8 and torch.cuda.is_bf16_supported() else torch.float16
+
+
+def build_model(path, args, device, dtype=None):
+    """dtype: torch dtype; None 时按 --dtype 解析 (auto: cuda 无 bf16 支持则 fp16, T4 即此路径)."""
+    dtype = dtype or resolve_dtype(args.dtype, device)
+    print(f"模型精度: {dtype}  (device={device})")
+    cfg = PonderLlamaConfig.from_llama(
+        LlamaConfig.from_pretrained(path),
+        ponder_signal="learned",                       # 训练必须用可学习信号
+        max_ponder_steps=args.max_steps,
+        ponder_epsilon=args.epsilon,
+        ponder_prior_p=args.prior_p,
+        ponder_loss_beta=args.beta,
+        ponder_start_layer=args.start_layer,
+        ponder_all_positions=True,                     # 训练时所有位置独立 halting
+    )
+    model = PonderLlamaForCausalLM.from_pretrained(
+        path, config=cfg, torch_dtype=dtype,
+        low_cpu_mem_usage=True, attn_implementation="sdpa")
+    # halting 头始终用 fp32 保证数值稳定
+    model.ponder_head = model.ponder_head.float()
+    # 初始停机概率: sigmoid(bias)。bias=-1 → λ≈0.27, 期望思考步数 ≈ 3.7
+    nn_init = torch.nn.init
+    with torch.no_grad():
+        model.ponder_head.bias.fill_(args.head_bias_init)
+    model.to(device)
+    if args.grad_ckpt and device != "cpu":
+        model.gradient_checkpointing_enable()
+    return model
+
+
+def setup_trainables(model, args):
+    """返回 (可训练参数组, 冻结说明). 三种模式对应不同的梯度开放范围."""
+    n = model.config.num_hidden_layers
+    s = model.config.ponder_start_layer
+    # 1) 全部冻结
+    for p in model.parameters():
+        p.requires_grad_(False)
+    groups = []
+    # 2) halting 头 (总是可训, fp32)
+    for p in model.ponder_head.parameters():
+        p.requires_grad_(True)
+        groups.append({"params": [p], "lr": args.lr_head, "name": "head"})
+    frozen_note = ["backbone"]
+    # 3) 思考块 / LoRA
+    if args.train_mode == "head+block":
+        for layer in model.model.layers[s:n]:
+            for p in layer.parameters():
+                p.requires_grad_(True)
+        groups.append({"params": [p for layer in model.model.layers[s:n]
+                                  for p in layer.parameters()],
+                       "lr": args.lr_block, "name": "block"})
+        frozen_note = ["layers 0.." + str(s - 1)]
+    elif args.train_mode == "head+lora":
+        try:
+            from peft import LoraConfig, get_peft_model
+        except ImportError as e:
+            raise SystemExit("head+lora 需要 peft: pip install peft") from e
+        scope = (list(range(s, n)) if args.lora_scope == "ponder" else list(range(n)))
+        target_names = set()
+        for i in scope:
+            for mod in ("q_proj", "k_proj", "v_proj", "o_proj"):
+                target_names.add(f"model.layers.{i}.self_attn.{mod}")
+            for mod in ("gate_proj", "up_proj", "down_proj"):
+                target_names.add(f"model.layers.{i}.mlp.{mod}")
+        lcfg = LoraConfig(r=args.lora_r, lora_alpha=args.lora_r * 2, lora_dropout=0.0,
+                          bias="none", task_type="CAUSAL_LM",
+                          target_modules=list(target_names))
+        model = get_peft_model(model, lcfg)
+        # 重新分组 (peft 包装后参数引用变化)
+        groups = [g for g in groups if g["name"] == "head"]
+        model.base_model.model.ponder_head.requires_grad_(True)  # 保险
+        lora_params = [p for pn, p in model.named_parameters()
+                       if p.requires_grad and "lora_" in pn]
+        groups.append({"params": lora_params, "lr": args.lr_lora, "name": "lora"})
+        frozen_note = [f"backbone (+LoRA r={args.lora_r} on {args.lora_scope} layers)"]
+    return model, groups, frozen_note
+
+
+# ========================================================================== #
+# 训练循环
+# ========================================================================== #
+def train_one_epoch(model, loader, optimizer, scheduler, device,
+                    accum=1, clip=1.0, log_every=0, global_step=0, scaler=None):
+    model.train()
+    stats = {"loss": 0.0, "ce": 0.0, "kl": 0.0, "steps": 0.0, "nb": 0}
+    optimizer.zero_grad(set_to_none=True)
+    t0 = time.time()
+    for it, (input_ids, labels, attn) in enumerate(loader):
+        input_ids, labels, attn = input_ids.to(device), labels.to(device), attn.to(device)
+        out = model(input_ids=input_ids, attention_mask=attn, labels=labels,
+                    output_ponder=True)
+        # 注意: forward 的 loss = CE + beta*KL; ponder_kl 单独带出便于监控
+        loss_to_back = out.loss / accum
+        if scaler is not None:
+            scaler.scale(loss_to_back).backward()   # fp16 梯度缩放防下溢
+        else:
+            loss_to_back.backward()
+        with torch.no_grad():
+            # 思考步数只统计有监督的位置
+            mask = labels[:, 1:] != -100
+            steps = out.ponder_steps[:, :-1][mask].float().mean() if mask.any() \
+                else out.ponder_steps.float().mean()
+            stats["loss"] += out.loss.item()
+            stats["ce"] += (out.loss.item()
+                            - (model.config.ponder_loss_beta * out.ponder_kl.item()))
+            stats["kl"] += out.ponder_kl.item()
+            stats["steps"] += steps.item()
+            stats["nb"] += 1
+        if (it + 1) % accum == 0 or (it + 1) == len(loader):
+            if scaler is not None:
+                scaler.unscale_(optimizer)          # clip 必须在真实梯度上做
+            torch.nn.utils.clip_grad_norm_(
+                [p for g in optimizer.param_groups for p in g["params"]], clip)
+            if scaler is not None:
+                scaler.step(optimizer)
+                scaler.update()
+            else:
+                optimizer.step()
+            scheduler.step()
+            optimizer.zero_grad(set_to_none=True)
+            global_step += 1
+            if log_every and global_step % log_every == 0:
+                k = max(stats["nb"], 1)
+                print(f"  step {global_step:4d} | loss {stats['loss']/k:7.4f} "
+                      f"(ce {stats['ce']/k:7.4f} + {model.config.ponder_loss_beta}·kl {stats['kl']/k:6.4f})"
+                      f" | ponder_steps {stats['steps']/k:5.2f} | lr {scheduler.get_last_lr()[0]:.2e}"
+                      f" | {time.time()-t0:6.1f}s", flush=True)
+                stats = {"loss": 0.0, "ce": 0.0, "kl": 0.0, "steps": 0.0, "nb": 0}
+                t0 = time.time()
+    return global_step
+
+
+def save_model(model, args, out_dir):
+    os.makedirs(out_dir, exist_ok=True)
+    core = model.get_base_model() if hasattr(model, "get_base_model") else model
+    # ponder_head 总是单独保存 (peft 适配器不包含它)
+    from safetensors.torch import save_file
+    save_file({"ponder_head.weight": core.ponder_head.weight.data.to(torch.float32).cpu(),
+               "ponder_head.bias": core.ponder_head.bias.data.to(torch.float32).cpu()},
+              os.path.join(out_dir, "ponder_head.safetensors"),
+              metadata={"note": f"signal=learned, K={args.max_steps}, p_g={args.prior_p}"})
+    if args.train_mode == "head+lora":
+        model.save_pretrained(out_dir)          # adapter_model.safetensors
+        print(f"已保存 LoRA 适配器 + ponder_head 到 {out_dir}")
+    elif args.train_mode == "head+block":
+        core.save_pretrained(out_dir)
+        print(f"已���存全量模型 (含新 ponder_head) 到 {out_dir}")
+    else:
+        print(f"已保存 ponder_head.safetensors 到 {out_dir} "
+              f"(推理时 from_ponder 加载后 model.ponder_head.load_state_dict)")
+
+
+# ========================================================================== #
+# main
+# ========================================================================== #
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[1])
+    ap.add_argument("--model", default=None,
+                    help="模型路径; 仅生成数据 (--make-demo-data/--make-mix-data) 时可省略")
+    ap.add_argument("--data", default=None, help="jsonl 训练数据; 不给且无 --make-* 时用内置演示数据")
+    ap.add_argument("--make-demo-data", default=None, metavar="PATH")
+    ap.add_argument("--make-mix-data", default=None, metavar="PATH",
+                    help="生成难度分级数据集 (easy/medium/hard + .eval.jsonl 评估集)")
+    ap.add_argument("--n-easy", type=int, default=220)
+    ap.add_argument("--n-medium", type=int, default=200)
+    ap.add_argument("--n-hard", type=int, default=180)
+    ap.add_argument("--n-eval-per-bucket", type=int, default=30)
+    ap.add_argument("--seed", type=int, default=7)
+    ap.add_argument("--text-field", default=None, help="纯文本字段名 (默认按 chat messages 解析)")
+    ap.add_argument("--train-mode", default="head", choices=["head", "head+lora", "head+block"])
+    ap.add_argument("--output", default="out/ponder")
+    ap.add_argument("--epochs", type=float, default=2.0)
+    ap.add_argument("--batch-size", type=int, default=2)
+    ap.add_argument("--accum", type=int, default=4)
+    ap.add_argument("--max-len", type=int, default=512)
+    ap.add_argument("--lr-head", type=float, default=1e-3)
+    ap.add_argument("--lr-lora", type=float, default=2e-4)
+    ap.add_argument("--lr-block", type=float, default=1e-5)
+    ap.add_argument("--warmup-ratio", type=float, default=0.03)
+    ap.add_argument("--clip", type=float, default=1.0)
+    ap.add_argument("--max-steps", type=int, default=8, help="思考步数上限 K")
+    ap.add_argument("--epsilon", type=float, default=0.01)
+    ap.add_argument("--prior-p", type=float, default=0.4, help="几何先验 p_g (越小越鼓励多想)")
+    ap.add_argument("--beta", type=float, default=0.05, help="KL 正则权重")
+    ap.add_argument("--head-bias-init", type=float, default=-1.0, help="初始停机偏置, sigmoid(-1)≈0.27")
+    ap.add_argument("--start-layer", type=int, default=None, help="思考块起始层 (默认最后8层)")
+    ap.add_argument("--lora-r", type=int, default=16)
+    ap.add_argument("--lora-scope", default="ponder", choices=["ponder", "all"])
+    ap.add_argument("--dtype", default="auto", choices=["auto", "float16", "bfloat16", "float32"],
+                    help="auto: T4 等 pre-Ampere 卡自动 fp16, 30/40/50 系 bf16, cpu fp32")
+    ap.add_argument("--grad-ckpt", action="store_true", default=None,
+                    help="思考块梯度重算 (16GB 显存训练强烈建议开启)")
+    ap.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")
+    ap.add_argument("--log-every", type=int, default=10)
+    args = ap.parse_args()
+
+    if args.make_mix_data:
+        make_mix_data(args.make_mix_data, n_easy=args.n_easy, n_medium=args.n_medium,
+                      n_hard=args.n_hard, n_eval_per_bucket=args.n_eval_per_bucket,
+                      seed=args.seed)
+        if not args.data:
+            return
+    if args.make_demo_data:
+        make_demo_data(args.make_demo_data)
+        if not args.data:
+            return
+    if not args.model:
+        ap.error("--model 为必填 (仅在仅使用 --make-demo-data / --make-mix-data 时可省略)")
+
+    # ---- 数据 ----
+    rows = load_jsonl(args.data) if args.data else DEMO_DATA
+    print(f"训练样本: {len(rows)} 条 (mode={args.train_mode})")
+    tok = AutoTokenizer.from_pretrained(args.model, trust_remote_code=True)
+    if tok.pad_token_id is None:
+        tok.pad_token = tok.eos_token
+    ds = PonderDataset(rows, tok, max_len=args.max_len, text_field=args.text_field)
+    loader = DataLoader(ds, batch_size=args.batch_size, shuffle=True,
+                        collate_fn=lambda b: collate(b, tok.pad_token_id or 1))
+
+    # ---- 模型 ----
+    dtype = resolve_dtype(args.dtype, args.device)
+    model = build_model(args.model, args, args.device, dtype=dtype)
+    model, groups, frozen_note = setup_trainables(model, args)
+    n_train = sum(p.numel() for g in groups for p in g["params"])
+    print(f"可训练参数: {n_train:,} ({[g['name'] for g in groups]}), 冻结: {frozen_note}")
+
+    # ---- 优化器 ----
+    optimizer = torch.optim.AdamW(
+        [{"params": g["params"], "lr": g["lr"]} for g in groups], weight_decay=0.01)
+    total_updates = max(1, int(len(loader) / args.accum * args.epochs))
+    scheduler = get_cosine_schedule_with_warmup(
+        optimizer, int(total_updates * args.warmup_ratio), total_updates)
+    # fp16 训���需要 GradScaler 防梯度下溢 (T4); bf16/fp32 不需要
+    scaler = None
+    if args.device.startswith("cuda") and dtype == torch.float16:
+        scaler = torch.amp.GradScaler(args.device)
+        print("已启用 GradScaler (fp16)")
+
+    # ---- 训练 ----
+    gs = 0
+    for ep in range(math.ceil(args.epochs)):
+        print(f"--- epoch {ep + 1}/{math.ceil(args.epochs)} ---")
+        gs = train_one_epoch(model, loader, optimizer, scheduler, args.device,
+                             accum=args.accum, clip=args.clip,
+                             log_every=args.log_every, global_step=gs, scaler=scaler)
+
+    save_model(model, args, args.output)
+    print("完成。推理加载示例:")
+    print("  model = PonderLlamaForCausalLM.from_ponder('<模型路径>',")
+    print(f"         ponder_kwargs={{'ponder_signal':'learned','max_ponder_steps':{args.max_steps}}})")
+    print("  head = load_file('<output>/ponder_head.safetensors')")
+    print("  model.ponder_head.load_state_dict(head)")
+
+
+if __name__ == "__main__":
+    main()