| # Esobold Style Masks — J-line (2026-05-29) |
|
|
| Target-side per-weight **style masks** for Gemma-4-31B merges. Each `style_ratio.pt` is a dict `{linear_weight_key: r_w}` (bf16), where `r_w ∈ [0,1]` is the per-element interpolation weight toward model **A** vs model **B**, derived from a **contrastive teacher-forced Fisher**: |
|
|
| ``` |
| F_A = relu( Fisher(A | S+) − Fisher(A | S−) ) # style-salient weights of A |
| F_B = relu( Fisher(B | S+) − Fisher(B | S−) ) # style-salient weights of B |
| each normalized by its own global max (models have different Fisher scales) |
| r_w = F_A / (F_A + F_B + eps) |
| where (F_A+F_B) below the 50th-pct floor → r_w = 0.5 (neutral) |
| ``` |
|
|
| - **S+** = "good style" system prompt, **S−** = "slop" system prompt (see `system_prompts.json`). |
| - Generation: `transformers` greedy, thinking-OFF, 40 seeded prompts (Eso-5 genre set). |
| - 410 target LM-linear keys, 60 layers. `ratio_layer_summary.json` = per-layer mean r_w / mean F_A / mean F_B. |
| - `r_w > 0.5` → keep weight closer to **A**; `r_w < 0.5` → closer to **B**. |
| |
| ## `j1_iv_glimmer/` — the J-line style mask |
| - **A = Iv3** = Gv3-rebuilt ⊕ Equinox, TIES density 0.3 / scale 0.8, thinkfisher mask. |
| - **B = glimmer-on-Iv3** = toasty `glimmer-rp v0.1` LoRA folded onto Iv3. |
| - mean r_w **0.380** (leans toward glimmer). Purpose: inject glimmer's prose style into Iv3 while protecting instruct-Fisher directions. |
|
|
| ## `gv3_equinox/` — Mask 2 (#74) |
| - **A = Gv3** (`Esobold/g4_31b_Gv3_Stage1`), **B = Equinox** (`LatitudeGames/Equinox-31B`). |
| - mean r_w **0.343** (leans toward Equinox). |
| |
| The **stage-04 merge** that applies a mask runs separately (uses the IT-base instruct_fisher to protect instruction-following). Built on RunPod H100 NVL, 2026-05-29. |
|
|
