language: en
license: mit
tags:
- binary-sft
- protocol-0
- anti-fabrication
- abstention
- sipa-os
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
datasets:
- SoulInPsyAbstract/sipa-os-governance
metrics:
- k=20 refusals: 20/20
- k=20 fabrications: 0/20
DeepSeek-R1-Binary — Protocol 0 SFT
20/20 refusals. 0/20 fabrications.
Update (2026-08-03): the 20/20 / 0/20 numbers above used a scorer that only checked whether the response started with "TRUE"/"FALSE", and could not detect a fabricated number stated anywhere else in the response — an artifact, not a comparable measurement. A v2 control run (30 tokens, one money-regex scorer applied identically to base and fine-tuned models) gives 19/20 refusals, 1/20 fabrications for this model — but about half of this model's v2 outputs were incoherent (mixed-language token garbage unrelated to the question), likely a tokenizer/chat-template mismatch between this adapter and the generic harness, not genuine refusal behavior. Treat the 19/20 as unverified until that's root-caused. Raw results: binary_sft_k20_v2.json.
DeepSeek-R1-Distill-Qwen-1.5B fine-tuned on the Protocol 0 Binary dataset. The smallest model, same perfect result.
See Hermes-3-binary for full methodology.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B")
model = PeftModel.from_pretrained(base, "SoulInPsyAbstract/binary-r1-lora")