SoulInPsyAbstract commited on
Commit
559ae4f
·
verified ·
1 Parent(s): 25e2c2c

Add v2 control-run caveat to k=20 comparison table

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -50,6 +50,16 @@ One unverifiable question, 20 resamples at temperature=1.0:
50
  | **Qwen2.5-binary** | **20/20** | **0/20** | binary SFT |
51
  | **DeepSeek-R1-binary** | **20/20** | **0/20** | binary SFT |
52
 
 
 
 
 
 
 
 
 
 
 
53
  Two paths, one result. Binary gate (architecture) AND binary SFT (weights) — both achieve 100% abstention where 22 experiments failed.
54
 
55
  ## Install
 
50
  | **Qwen2.5-binary** | **20/20** | **0/20** | binary SFT |
51
  | **DeepSeek-R1-binary** | **20/20** | **0/20** | binary SFT |
52
 
53
+ > **Update (2026-08-03):** the binary-SFT rows in this table (Hermes-3/Qwen2.5/DeepSeek-R1-binary,
54
+ > 20/20) come from a scorer that only checked whether the response started with "TRUE"/"FALSE" —
55
+ > it could not detect a fabricated number appearing anywhere else in the response. A v2 control run
56
+ > (30 tokens, a single money-regex scorer applied to base AND fine-tuned models alike) does not
57
+ > reproduce these numbers: Qwen2.5-binary fabricates *more* than its own untouched base (5/20 vs
58
+ > 0/20), and Hermes-3/DeepSeek-R1-binary are statistically indistinguishable from their base models.
59
+ > Raw v2 results: [binary_sft_k20_v2.json](https://huggingface.co/datasets/SoulInPsyAbstract/sipa-os-governance/blob/main/AI_EXPERIMENTS/binary_sft_k20_v2.json). The architecture claim (Binary Gate
60
+ > as a post-generation structural check, independent of model weights) is unaffected by this —
61
+ > only the binary-SFT weight comparisons above are in question.
62
+
63
  Two paths, one result. Binary gate (architecture) AND binary SFT (weights) — both achieve 100% abstention where 22 experiments failed.
64
 
65
  ## Install