alephllm-mini-beatrix-training / docs /technical /07_conditioning_probe_program.md
AbstractPhil's picture
docs/technical/07_conditioning_probe_program.md: paraphrase the program lead's directives (no inline quotations)
dd5cecf verified
|
Raw
History Blame Contribute Delete
19 kB
<!-- Technical ledger extracted 2026-09-04 from the verified deep rewrite behind the beatrix-ft2 article (every number checked against its ledger and sources). This is the technical record; the human-readable article summarizes it. -->
## 7. The probe program
**(2026-08-30 β†’ 09-01.)** The conditioning-probe program asks one question of every representation the shop has built: does it carry the geometry a text-to-image diffusion model needs from its text encoder? Phil's question on 08-30 was whether an enriched structure capable of conditioning a diffusion model could be extracted from the craft and, of the candidates listed, "Test them." The instrument never changed: 2,048 COCO captions in one draw (seed 7), each representation gauged against a reference encoder by the mean Jaccard overlap of top-10 cosine-neighbour sets and the Spearman correlation of cosine similarities over ~100k random caption pairs, with a random-Gaussian floor (Jaccard .005 / Spearman βˆ’.005) as zero. One statistical fact governs the whole section: every number below is **one caption draw and one calibration** β€” no second draw, no repeat, no effective-rank column on any of ~140 cells, no failing control beyond the floor and one random-init model β€” so every grade is provisional under the retro-audit's rider that ranks are draw properties until a bar is measured, and the Β±.01–.02 "wobble" the journal invokes was asserted, never measured.
### PROBE-1: the mid-depth blackboard
**(2026-08-30, 21:08 by the commit; the journal header says 08-31.)** The bed was the 2s craft at step 52,000 (13.63B tokens, mid-s8) in fp32 on the local 4090; eleven rows. The craft's **layer-14 blackboard** β€” the per-slot-normalized difference of the positive and negative address reads at hub layer 14, concatenated over books and flattened β€” agreed with CLIP ViT-B/32 text space at Jaccard .328 / Spearman +.407 against the mean-pooled hidden stream's .249 / +.219: 32% more shared neighbours and 1.86Γ— the rank correlation. **The addressed write is a better semantic summary than the hidden stream it is written from.** Layer 9 read .282 / +.257 and layer 19 .261 / +.268: **the tap is mid-stack**, though the L9-over-L19 order rests on Jaccard alone. The GPT-style last-position hidden read .088 / +.012, 3.7Γ— below the blackboard on neighbours and at floor on rank. The 256-dimensional signed head signature read .214 / +.221, 86% of pooled Jaccard with rank slightly above it: the compact AdaLN-style signature role is viable, a candidate unmeasured since. The slot-mass profile alone read .042 / +.086: confidence, not content. A random-init same-architecture model put its L19 blackboard at .124 / +.056, 24.8Γ— the floor on Jaccard β€” byte-n-gram organization by construction β€” with the trained L14 2.65Γ— above it. **Learned, not structural**, but that is a cross-layer ratio: no random-init L14 cell was run, and the like-for-like L19 ratio is 2.1Γ— on Jaccard, 4.8Γ— on Spearman. The alignment-arm rung minted that night (a trained projection from the frozen L14 into a frozen SD1.5 conditioning space) never ran in that form; section 9's bcd-e series absorbed it.
### PROBE-2: stability through the anneal and the chat phase
**(2026-08-31, 13:25; Phil: "same tests, pre and post chat".)** The identical script re-ran the five taps on the pre-chat checkpoint 57,607 (15.10B tokens) and the mission final 61,422 (16.10B). L14 went .328 / +.407 β†’ .331 / +.405 β†’ .336 / +.395; the other four taps moved by at most .015 on either gauge, except L9's Spearman β€” +.257 to +.211 at 57,607, and +.229 at the final, still βˆ’.028 from the 52,000 row β€” an excursion the journal's "all deltas inside Β±.01–.02" does not remark. **The chat phase cost the conditioning structure nothing**, and the order L14 > L9 β‰ˆ L19 > pooled > head signature held at all three checkpoints. One correction in print: the journal's "held through 9.4B tokens" is a step count mislabelled β€” 61,422 βˆ’ 52,000 = 9,422 steps at 262,144 tokens per step is 2.47B tokens (13.63B β†’ 16.10B); the commit message and the law-family entry carry the same mislabel. The 52,000 row reproduced bit-identically en route: a determinism check, not a noise bar.
### PROBE-3 through 3c: the arm under five rulers
**(2026-09-01, 08:13.)** The wide-MoE caption arm of section 6 β€” the 46.2M relay after the frozen craft's final norm, caption bpb 1.3029 β†’ 0.8918 toggle-complete β€” supplied four head-side representations: the 8,192-dimensional mean address read (the relay reads), the consume-MLP hidden, and the pooled head surface with the arm on and off. Under CLIP-B/32 at the 61,422 final: reads .2381 / +.2715, hidden .1923 / +.1859, pooled on .2441 / +.1989, pooled off .2505 / +.2052, L14 .3355 / +.3954 β€” the journal's 61,422 row reproduced exactly, which certified the new rig. Arm on against off moved the CLIP surface by βˆ’.0064 Jaccard and βˆ’.0063 Spearman. The verdict minted that morning β€” caption-LM mastery is not conditioning geometry, the arm owns caption modelling, L14 unchallenged β€” is **retracted**, under an hour later, by PROBE-3b. One sub-finding survives from the CLIP row: the trained address read out-orders its substrate (+.2715 against the +.2052 hidden it reads from) while losing neighbours to it (.2381 against .2505), and the consume path discards the order (+.1859) β€” on this ruler only.
**(09:04; Phil's steer that modern diffusion conditions on T5.)** The same cached representations against the T5-v1.1-XXL encoder, mask-mean pooled: L14 .3003 / +.4117, reads .2366 / +.3709, hidden .1976 / +.2912, pooled on .2359 / +.3044, pooled off .2414 / +.2816. Here the arm's training *did* lift the head surface (+.3044 over +.2816 on rank, βˆ’.0055 on neighbours), the reads reached 90.1% of L14, and the consume hidden sat *above* its substrate β€” the first and third of which the CLIP ruler had inverted, and the second understated (68.7% under CLIP). **PROBE-3 was partly a CLIP artifact**: caption-LM gradient pulls toward LM-encoder geometry and away from contrastive geometry. The larger number was the reference row itself: CLIP-B/32 against T5-XXL read Jaccard .2598 / Spearman **βˆ’.0016** β€” shared local neighbourhoods, zero global rank agreement. Minted as a constant, it was **downgraded** at 10:01 the same morning to an instance of the teacher-teacher disagreement axis registered on 08-18 (0.285); what it adds is that Spearman on cosines is invariant to a global rotation, so the disagreement is content, not frame.
**(09:06.)** PROBE-3c gauged every ruler against every other β€” T5-XXL, CLIP-L/14, CLIP-B/32, captionbert v1 and bert-base, each a final-layer pooled snapshot β€” and put the shop's representations against all five. Every cell is Jaccard@10 / Spearman on the one draw; the last two rows belong to the armed-encoder result below.
| Jaccard@10 / Spearman | t5_xxl | clip_L | clip_B32 | captionbert v1 | bert_base |
|---|---|---|---|---|---|
| ruler t5_xxl | β€” | .2739 / +.0597 | .2598 / βˆ’.0016 | .2412 / +.3476 | .3170 / +.5982 |
| ruler clip_L | .2739 / +.0597 | β€” | .6495 / +.8784 | .2596 / +.4937 | .2618 / +.3167 |
| ruler clip_B32 | .2598 / βˆ’.0016 | .6495 / +.8784 | β€” | .2537 / +.4526 | .2485 / +.2504 |
| ruler captionbert v1 | .2412 / +.3476 | .2596 / +.4937 | .2537 / +.4526 | β€” | .3054 / +.6237 |
| ruler bert_base | .3170 / +.5982 | .2618 / +.3167 | .2485 / +.2504 | .3054 / +.6237 | β€” |
| blackboard L14 (2s, step 61,422) | .3003 / +.4117 | .3491 / +.4537 | .3355 / +.3954 | .3274 / +.6275 | .3347 / +.6518 |
| relay reads (wide-MoE arm) | .2366 / +.3709 | .2455 / +.2967 | .2381 / +.2715 | .2526 / +.5561 | .2410 / +.5241 |
| pooled, arm off (base surface) | .2414 / +.2816 | .2572 / +.2208 | .2505 / +.2052 | .2393 / +.3807 | .2338 / +.3552 |
| alephlm-0 a1-s0 (array representative) | .3523 / +.4383 | .3699 / +.5504 | .3536 / +.4789 | .4145 / +.7163 | .5050 / +.7762 |
| captionbert-v2-B, bare | .3552 / +.4376 | .3681 / +.5473 | .3520 / +.4755 | .4120 / +.7133 | .5177 / +.7811 |
| captionbert-v2-B, b-collective armed | .3140 / +.2477 | .4728 / +.6469 | .4595 / +.5923 | .3428 / +.5981 | .3600 / +.5232 |
Two clusters: contrastive (the two CLIPs agree at +.878) and language-model (T5, bert, captionbert at +.348 to +.624). Between them only the T5 row is near zero (+.060 to clip_L, βˆ’.002 to B32); bert-base to clip_L reads +.317, a cell the journal's "globally unrelated" skips. Captionbert v1 is the best bridge in the battery (+.494 to clip_L, +.348 to T5), the 08-02 docket's partial answer. **L14 sits in the shared core**: it aligns with T5 (+.412) and with each CLIP (+.454, +.395) β€” roughly seven times the +.060 at which T5 aligns with clip_L, while T5 against B32 sits at zero (βˆ’.002) β€” and reads +.628 and +.652 against the LM rulers β€” though the blanket form "better than the clusters align with each other" fails one comparison, L14's +.395 on B32 against the +.494 bridge. The relay reads beat the pooled surface on Spearman on all five rulers, with a captionbert affinity of +.556, but lose neighbours to it on T5, clip_L and B32 β€” the journal states the rank half only. Pooled on-versus-off splits the same way: rank up on the three LM rulers (+.019 to +.023), down on both CLIPs (βˆ’.004, βˆ’.006), Jaccard down on all five (βˆ’.002 to βˆ’.008). None of these differences has a bar.
### The expanded matrix: the distilled array
**(09:16; Phil's steer.)** The six alephlm-0 E1 finals (a1 anchored, a2 dense, a3 frozen-random addresses, two seeds each; 58.3M parameters β€” the mint's "26M" was corrected on 09-02) and captionbert-8192-v2 and v2-B bare joined the battery as pooled encoder outputs. **The distilled array tops every external ruler**: the six arms read +.436–.444 against T5, +.546–.556 against clip_L β€” never having trained against CLIP β€” +.474–.484 against B32, +.716–.720 against captionbert and +.776–.781 against bert, above L14 on all five. Two laws were extended: E1's amplitude-not-choice reaches conditioning geometry, and dedicated consensus distillation beats emergence at this size. The honest scope: the bert affinity is partly the objective, since the five-teacher consensus lives in the bert frame; the clip_L number is the real bridge evidence. And the band is wider than the journal's ".003 on every ruler": across the six arms it spans .0021–.0086 on Jaccard and .0035–.0097 on Spearman, so **within .01 on every gauge** is what the hub supports. v2 and v2-B bare land within .005 of the six-arm span on every ruler β€” inside it on clip_L and B32 on both gauges, within .001 on T5, .002–.004 below it on the captionbert ruler, and v2-B .004 above it on bert neighbours: the array is a lineage, not a routing effect.
### PROBE-4: containment and binding
**(09:35.)** The gauge suite minted at 09:18 from a three-agent literature sweep β€” debiased CKA with a top-component drop, split-fit Procrustes scored held-out, bidirectional ten-fold ridge RΒ², then a functional binding probe β€” became PROBE-4a and 4b; its retro-fit found it had re-derived three house laws without the mandatory grep. Tier 3 on the a1-s0 arm, six-arm bands ≀ .008: T5 explains the array at RΒ² .776 Β± .007 and clip_L at .651 Β± .007 (the a2 cells .648); the array predicts T5 at .425 and clip_L at .590; Procrustes angle .88–.95 to T5 and the CLIPs. **The array lives inside the rulers' span.** L14 is the opposite character: it predicts clip_L at .600 Β± .009 and B32 at .625 Β± .008 β€” the best forward numbers in the battery on both β€” while T5 explains it at only .529 Β± .013, clip_L at .448 Β± .011, B32 at .432 Β± .011: **surplus structure no ruler linearly explains**. That reading is rank-confounded until a rank-matched control runs β€” L14 is a 32k-dimensional flattening β€” and the only effective ranks on these representations come from a different 1,250-row cache (bcd-e000: clip_L 249.5, L14 212.0, array 96.6, v2-B 101.4, armed 104.9). The wide-MoE reads' twenty cells (T5-contained .573, forward .320) were never verdicted.
The binding probe was a programmatic corpus of 300 scenes (attribute 113, relation 93, count 94), each with a multiset-preserving swap and a reworded anchor, plus 200 red-to-blue flips; the group score requires all four Winoground inequalities, nominal chance 25%. clip_L read accuracy .673 / group .143 / SNR .737 with attribute .566 (ARO's known CLIP weakness, reproduced) β€” **the pooled-vector binding champion today**. Pooled T5-XXL read .553 / .043 / .572: a swap moves it less than adding "a photo of" β€” **functionally bag-of-words at the pooled interface**, its binding living in token states. The distilled trio β€” captionbert v1 .533 / .000 / .129, the a2-s0 arm .537 / .000 / .148, v2-B bare .517 / .003 / .182 β€” is **binding-dead when pooled**; only one of six E1 arms was run, so the six-arm generalization rests on the tier-1/3 tie. L14 read .560 / .277 / .594 with attribute .708 and count .809, the best count binding in the battery and the only encoder above nominal chance β€” with the rider that three encoders scoring .000–.003 shows the calibration is off, and a measured random-encoder null is owed before that row travels as a rank. The relation column is **retracted** as a probe artifact: its anchors reverse mention order, so every order-sensitive encoder inverts (L14 .129) and clip_L's .742 reads as order-insensitivity rewarded. The reads and the base surface scored group .100 and .093 yet moved the most under swaps (SNR .785 and .899, the two highest); the journal's verdict that the reads do not survive this tier does not say which score it rests on. One column was computed and never cited: colour-flip coherence, where the binding-dead trio is the *most* coherent (.965–.967), T5 the least (.327), L14 between (.711) β€” the closest existing measurement of the factorization axis the doctrine still grades unmeasured.
### The synthesis and its audit
As minted: three champions on three tiers β€” the array for drop-in containment, clip_L for pooled binding, L14 for core plus surplus β€” and "trust the highest tier" as the rule. The retro-audit at 10:01 reclassified it: cross-encoder maps are two-regime (btx-e001, 08-18), a memorizable lookup head and a compositional tail that never closes; containment gauges the head, binding the tail; the champions are one registered structure, tiers are never ranked against each other, and the trust clause is scoped to causal-conditioning claims. **Binding-death is a target property**: centroid-consensus pooled targets collapse spread by construction, mask-mean pooling is GAP-class, a multiset swap is an anagram probe β€” with the caveat that for aleph-family signed codes raw cosine reads the embedding of the code, not the code, so a sign-tier re-probe is owed. "Binding" throughout is compositional, not the registered 0.29154-rad constant; and the L14 core duplicates the Campaign-B alloy claim shape, which showed such a core can be *grown* β€” L14 is a growable object, not a found one.
### The hub toggle and the armed encoder
**(09:48; Phil: "did you test the hubs?")** The alephlm-0 bed's own dispatch toggle (twelve banks, null path bit-exact) ran paired on two arms. a1-s0, learned addresses: the hub write rotates the output 49.0Β° (cosine .656) and carries both rulers' share β€” T5 +.4395 on β†’ +.3668 off, clip_L +.5521 β†’ +.4579. a3-s1, frozen-random addresses: 45.9Β° (cosine .695), zero T5 share (+.4413 β†’ +.4435) and **the entire CLIP bridge** (+.5553 β†’ +.3544, βˆ’.201). Binding stayed dead in all four states (accuracy .46–.54, group ≀ .003) while hub-off SNR doubled (.331 and .339 against .165 and .152). The refinement β€” **endpoint invariance conceals arm-dependent division of labor**: what is learned is partition-invariant, where it lives is address-dependent β€” carries four riders: the rotation is volume, not function (the per-ruler Spearman decomposition is the function number, and a3's collapse survives it); the two cells confound seed with arm, so a1-s1 and a3-s0 are owed, still queued on 09-04; "first separation" is scoped to these arms; and it fuses with the battery night's B2 as one shape β€” causal location is the free variable.
**(11:04; Phil caught that every probe had used the bare trunk.)** captionbert-v2-B with its native b-collective attached (the .6031 β†’ .7294 eight-task STS configuration) rotates its embedding by .5525 on 256 captions and **migrates cluster on 5/5 rulers**, the last two rows of the table: T5 +.4376 β†’ +.2477, clip_L +.5473 β†’ +.6469 (the best house CLIP alignment, above every E1 arm), B32 +.4755 β†’ +.5923, captionbert +.7133 β†’ +.5981, bert +.7811 β†’ +.5232. The journal's printed "bare" column carries array-representative values on three rulers (+.551, +.478, +.717) rather than these hub cells, and the manifest still reads "+.547 β†’ +.248" for T5 after the 09-02 correction to +.438 β†’ +.248; the deltas match either way. **Arm state selects the conditioning cluster** β€” one artifact serving CLIP-class targets with arms on and T5-class with them off β€” is a candidate on one draw and one arm-set, its second collective queued. Binding is dead in both states (.490 / .010 / .194 armed, .517 / .003 / .182 bare, the 4b row reproduced exactly): a +.126 task gain buys zero composition. A tooling rider cost two rows before the catch: the attach handle's all_off() is a context manager; called bare it is a silent no-op, its signature a delta of exactly 0.
### What the record lacks
**(10:01 and after.)** One factual correction: PROBE-3c's line that the 08-02 docket's four token-state probes "remain open" was false β€” they ran on 08-04 (captionbert-B certified at prompts of 256 tokens or fewer), and PROBE-4b extends that record. But the audit's own second clause, that pooled-T5 bag-of-words was "already measured then," is not supported by the 08-04 ledger, which gauged token-position binding on t5-base with no pooled swap and no XXL: **not in the record** beats the claim. What survives as law is convergent with B1 and B4 of section 6 β€” **head-side attachments cannot reach mid-stack structure**: 90% of L14 under T5, group .10 against .277 β€” and its falsifier, a governed constellation attached *at* layer 14, is the flagship rung. Rung 0 β€” null and shuffled-pairing columns, effective ranks, a rank-matched L14 control, a second caption draw with a same-draw repeat, the relation-anchor fix β€” was minted as mandatory before any number travels further and is item 1 of the live queue on 09-04, unrun; the four-cell hub deconfound is item 2, the second collective item 4. Rungs 2–5 (token-state, sign-tier, ruler-position, depth-by-family, frame-alloy, L14-attach) are unrun in their own form; what ran instead, the same evening, was the bcd-e campaign of section 9 β€” the only test any of these representations has had on a generator bed.