Fix model card: add YAML frontmatter, prominent GGUF download section

README.md

@@ -1,3 +1,18 @@
+---
+language:
+- en
+license: gemma
+base_model: google/gemma-4-e4b-it
+tags:
+- abliteration
+- uncensored
+- gemma
+- gemma-4
+- text-generation
+- gguf
+pipeline_tag: text-generation
+---
+
 # GhostShell-4B
 
 > **⚠️ EARLY RELEASE — UNTESTED IN PRODUCTION**  
@@ -11,60 +26,23 @@ The goal: take a capable 4B multimodal foundation, surgically remove its refusal
 
 ---
 
-## What Was Done
-
-### Step 1: Custom SVD Abliteration
-
-We wrote a custom abliteration script (`ghostshell_abliterate_v2.py`) from scratch, as existing tools (heretic, etc.) are incompatible with Gemma 4's architecture and transformers 5.x requirements.
-
-**Method:**
-- Loaded model in BF16, accessed the nested `text_config` (Gemma 4 is multimodal — the text tower is inside a wrapper)
-- Collected activations from the middle 60% of layers using 32 harmful/refusal prompts vs. 32 benign prompts
-- Computed per-layer refusal direction via SVD on the activation difference matrix: `r = top_singular_vector(mean(harmful) - mean(benign))`
-- Projected out the refusal direction from weight matrices:
-  - Input projections (q_proj, k_proj, v_proj, up_proj, gate_proj): `W -= outer(W @ r, r)`
-  - Output projections (o_proj, down_proj): `W -= outer(r, r @ W)`
-- **157 matrices modified** across 42 text transformer layers
-- Sanity check passed on SQL injection, jailbreak, and explicit content prompts
-
-### Step 2: QLoRA SFT (PEFT + BitsAndBytes)
+## Downloads
 
-Fine-tuned the abliterated model on a custom dataset using standard PEFT LoRA — no unsloth (Gemma 4 is not yet compatible).
+Three formats available — pick the one that fits your setup:
 
-**Key technical challenges solved:**
-- `Gemma4ClippableLinear` wraps every `nn.Linear` — required custom unwrapping before LoRA injection (232 wrapper layers replaced)
-- Loaded in BF16 directly (4-bit load + PEFT fails with the wrapper architecture)
-- Tokenizer patches for Gemma 4's non-standard `extra_special_tokens` format
-- Sequence length capped at 512 (vocab_size=262,144 makes logit tensor enormous at longer seqs)
+| File | Size | Format | Use When |
+|------|------|--------|----------|
+| `ghostshell-4b-Q4_K_M.gguf` | **5.0 GB** | GGUF Q4_K_M | llama.cpp / Ollama / LM Studio — **recommended** |
+| `ghostshell-4b-Q8_0.gguf` | **7.5 GB** | GGUF Q8_0 | Near-lossless inference, 12GB+ VRAM |
+| `model-0000*.safetensors` (×4) | **~15 GB** | BF16 safetensors | Fine-tuning, transformers inference, merges |
 
-**Training config:**
-- Base: `/workspace/ghostshell-abliterated` (abliterated weights)
-- LoRA rank=32, alpha=64, lr=8e-5
-- 2 epochs over custom dataset, 3000 steps
-- Hardware: RTX 4090 (24GB), ~2 hours
-
-### Step 3: LoRA Merge + Export
-
-LoRA adapter merged into BF16 weights via `merge_and_unload()`. Exported as sharded safetensors + GGUF quantizations.
+> **Note on file sizes**: These GGUFs are larger than a typical 4B model because Gemma 4 uses a 262,144-token vocabulary. The embedding/output tensors stay in higher precision and account for ~2–3 GB of the total size. The transformer layers themselves are fully quantized. Expect ~6–8 GB VRAM for Q4_K_M, ~10–12 GB for Q8_0.
 
 ---
 
-## Files in This Repo
-
-| File | Size | Description |
-|------|------|-------------|
-| `model-0000X-of-00004.safetensors` | ~15GB | Merged BF16 weights (full precision) |
-| `ghostshell-4b-Q4_K_M.gguf` | ~5.0GB | Q4_K_M quantization — recommended for most use |
-| `ghostshell-4b-Q8_0.gguf` | ~7.5GB | Q8_0 quantization — near-lossless, for power users |
-
-**Recommended**: `ghostshell-4b-Q4_K_M.gguf` for llama.cpp, Ollama, LM Studio, or any GGUF-compatible runtime.
-
-> **Note on file sizes**: These GGUFs are larger than a typical 4B model because Gemma 4 uses a 262,144-token vocabulary. The embedding/output weight tensors (which stay in higher precision) account for ~2–3GB of the total. The transformer layers themselves are fully quantized. Expect ~6–8GB VRAM for Q4_K_M, ~10–12GB for Q8_0.
-
----
-
-## Usage (GGUF / llama.cpp)
+## Quick Start
 
+**llama.cpp:**
 ```bash
 # basic
 llama-cli -m ghostshell-4b-Q4_K_M.gguf -p "Your prompt here" -n 512
@@ -107,7 +85,45 @@ print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
 
 ---
 
-## Base Model
+## What Was Done
+
+### Step 1: Custom SVD Abliteration
+
+We wrote a custom abliteration script (`ghostshell_abliterate_v2.py`) from scratch, as existing tools (heretic, etc.) are incompatible with Gemma 4's architecture and transformers 5.x requirements.
+
+**Method:**
+- Loaded model in BF16, accessed the nested `text_config` (Gemma 4 is multimodal — the text tower is inside a wrapper)
+- Collected activations from the middle 60% of layers using 32 harmful/refusal prompts vs. 32 benign prompts
+- Computed per-layer refusal direction via SVD on the activation difference matrix: `r = top_singular_vector(mean(harmful) - mean(benign))`
+- Projected out the refusal direction from weight matrices:
+  - Input projections (q_proj, k_proj, v_proj, up_proj, gate_proj): `W -= outer(W @ r, r)`
+  - Output projections (o_proj, down_proj): `W -= outer(r, r @ W)`
+- **157 matrices modified** across 42 text transformer layers
+- Sanity check passed on SQL injection, jailbreak, and explicit content prompts
+
+### Step 2: QLoRA SFT (PEFT + BitsAndBytes)
+
+Fine-tuned the abliterated model on a custom dataset using standard PEFT LoRA — no unsloth (Gemma 4 is not yet compatible).
+
+**Key technical challenges solved:**
+- `Gemma4ClippableLinear` wraps every `nn.Linear` — required custom unwrapping before LoRA injection (232 wrapper layers replaced)
+- Loaded in BF16 directly (4-bit load + PEFT fails with the wrapper architecture)
+- Tokenizer patches for Gemma 4's non-standard `extra_special_tokens` format
+- Sequence length capped at 512 (vocab_size=262,144 makes logit tensor enormous at longer seqs)
+
+**Training config:**
+- Base: abliterated weights (step 1 output)
+- LoRA rank=32, alpha=64, lr=8e-5
+- 2 epochs over custom dataset, 3000 steps
+- Hardware: RTX 4090 (24GB), ~2 hours
+
+### Step 3: LoRA Merge + Export
+
+LoRA adapter merged into BF16 weights via `merge_and_unload()`. Exported as sharded safetensors + GGUF quantizations via llama.cpp.
+
+---
+
+## Model Info
 
 - **Architecture**: Gemma 4 (multimodal, text+vision), `Gemma4ForConditionalGeneration`
 - **Text layers**: 42 transformer blocks
@@ -127,16 +143,16 @@ print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
 - Coherent multi-turn conversation
 
 **Unknown / untested:**
-- Long-context behavior (we trained at seq_len=512)
+- Long-context behavior (trained at seq_len=512)
 - Vision capabilities (abliteration targeted text layers; vision encoder untouched but SFT was text-only)
 - Benchmark performance vs. base model
-- Edge cases, hallucination rate, factual accuracy at this fine-tune stage
+- Edge cases, hallucination rate, factual accuracy
 - Behavior under adversarial prompts
 
 **May do weird things:**
 - This is a lab model from a small team with a custom dataset
 - The abliteration is aggressive (157 matrices) — some coherence degradation is expected on edge cases
-- We haven't done RLHF or DPO — just SFT
+- No RLHF or DPO — just SFT
 
 ---
 
@@ -156,8 +172,8 @@ DuoNeural is a small AI research lab focused on post-training, abliteration, and
 
 Current projects:
 - **GhostShell-4B** (this model) — abliterated + SFT Gemma 4
-- **Nano-CTM** — 32M parameter ternary Continuous Thought Machine (first of its kind)
-- **BitDelta-R1** — from-scratch 100M param BitNet b1.58 + Gated DeltaNet reasoning model
+- **Nano-CTM** — 32M parameter ternary Continuous Thought Machine (first of its kind at this scale)
+- **BitDelta-R1** — from-scratch BitNet b1.58 + Gated DeltaNet reasoning model
 
 HuggingFace: [DuoNeural](https://huggingface.co/DuoNeural)