alephllm-mini-beatrix-training / docs /technical /07_conditioning_probe_program.md
AbstractPhil's picture
docs/technical/07_conditioning_probe_program.md: paraphrase the program lead's directives (no inline quotations)
dd5cecf verified
|
Raw
History Blame Contribute Delete
19 kB

7. The probe program

(2026-08-30 β†’ 09-01.) The conditioning-probe program asks one question of every representation the shop has built: does it carry the geometry a text-to-image diffusion model needs from its text encoder? Phil's question on 08-30 was whether an enriched structure capable of conditioning a diffusion model could be extracted from the craft and, of the candidates listed, "Test them." The instrument never changed: 2,048 COCO captions in one draw (seed 7), each representation gauged against a reference encoder by the mean Jaccard overlap of top-10 cosine-neighbour sets and the Spearman correlation of cosine similarities over ~100k random caption pairs, with a random-Gaussian floor (Jaccard .005 / Spearman βˆ’.005) as zero. One statistical fact governs the whole section: every number below is one caption draw and one calibration β€” no second draw, no repeat, no effective-rank column on any of ~140 cells, no failing control beyond the floor and one random-init model β€” so every grade is provisional under the retro-audit's rider that ranks are draw properties until a bar is measured, and the Β±.01–.02 "wobble" the journal invokes was asserted, never measured.

PROBE-1: the mid-depth blackboard

(2026-08-30, 21:08 by the commit; the journal header says 08-31.) The bed was the 2s craft at step 52,000 (13.63B tokens, mid-s8) in fp32 on the local 4090; eleven rows. The craft's layer-14 blackboard β€” the per-slot-normalized difference of the positive and negative address reads at hub layer 14, concatenated over books and flattened β€” agreed with CLIP ViT-B/32 text space at Jaccard .328 / Spearman +.407 against the mean-pooled hidden stream's .249 / +.219: 32% more shared neighbours and 1.86Γ— the rank correlation. The addressed write is a better semantic summary than the hidden stream it is written from. Layer 9 read .282 / +.257 and layer 19 .261 / +.268: the tap is mid-stack, though the L9-over-L19 order rests on Jaccard alone. The GPT-style last-position hidden read .088 / +.012, 3.7Γ— below the blackboard on neighbours and at floor on rank. The 256-dimensional signed head signature read .214 / +.221, 86% of pooled Jaccard with rank slightly above it: the compact AdaLN-style signature role is viable, a candidate unmeasured since. The slot-mass profile alone read .042 / +.086: confidence, not content. A random-init same-architecture model put its L19 blackboard at .124 / +.056, 24.8Γ— the floor on Jaccard β€” byte-n-gram organization by construction β€” with the trained L14 2.65Γ— above it. Learned, not structural, but that is a cross-layer ratio: no random-init L14 cell was run, and the like-for-like L19 ratio is 2.1Γ— on Jaccard, 4.8Γ— on Spearman. The alignment-arm rung minted that night (a trained projection from the frozen L14 into a frozen SD1.5 conditioning space) never ran in that form; section 9's bcd-e series absorbed it.

PROBE-2: stability through the anneal and the chat phase

(2026-08-31, 13:25; Phil: "same tests, pre and post chat".) The identical script re-ran the five taps on the pre-chat checkpoint 57,607 (15.10B tokens) and the mission final 61,422 (16.10B). L14 went .328 / +.407 β†’ .331 / +.405 β†’ .336 / +.395; the other four taps moved by at most .015 on either gauge, except L9's Spearman β€” +.257 to +.211 at 57,607, and +.229 at the final, still βˆ’.028 from the 52,000 row β€” an excursion the journal's "all deltas inside Β±.01–.02" does not remark. The chat phase cost the conditioning structure nothing, and the order L14 > L9 β‰ˆ L19 > pooled > head signature held at all three checkpoints. One correction in print: the journal's "held through 9.4B tokens" is a step count mislabelled β€” 61,422 βˆ’ 52,000 = 9,422 steps at 262,144 tokens per step is 2.47B tokens (13.63B β†’ 16.10B); the commit message and the law-family entry carry the same mislabel. The 52,000 row reproduced bit-identically en route: a determinism check, not a noise bar.

PROBE-3 through 3c: the arm under five rulers

(2026-09-01, 08:13.) The wide-MoE caption arm of section 6 β€” the 46.2M relay after the frozen craft's final norm, caption bpb 1.3029 β†’ 0.8918 toggle-complete β€” supplied four head-side representations: the 8,192-dimensional mean address read (the relay reads), the consume-MLP hidden, and the pooled head surface with the arm on and off. Under CLIP-B/32 at the 61,422 final: reads .2381 / +.2715, hidden .1923 / +.1859, pooled on .2441 / +.1989, pooled off .2505 / +.2052, L14 .3355 / +.3954 β€” the journal's 61,422 row reproduced exactly, which certified the new rig. Arm on against off moved the CLIP surface by βˆ’.0064 Jaccard and βˆ’.0063 Spearman. The verdict minted that morning β€” caption-LM mastery is not conditioning geometry, the arm owns caption modelling, L14 unchallenged β€” is retracted, under an hour later, by PROBE-3b. One sub-finding survives from the CLIP row: the trained address read out-orders its substrate (+.2715 against the +.2052 hidden it reads from) while losing neighbours to it (.2381 against .2505), and the consume path discards the order (+.1859) β€” on this ruler only.

(09:04; Phil's steer that modern diffusion conditions on T5.) The same cached representations against the T5-v1.1-XXL encoder, mask-mean pooled: L14 .3003 / +.4117, reads .2366 / +.3709, hidden .1976 / +.2912, pooled on .2359 / +.3044, pooled off .2414 / +.2816. Here the arm's training did lift the head surface (+.3044 over +.2816 on rank, βˆ’.0055 on neighbours), the reads reached 90.1% of L14, and the consume hidden sat above its substrate β€” the first and third of which the CLIP ruler had inverted, and the second understated (68.7% under CLIP). PROBE-3 was partly a CLIP artifact: caption-LM gradient pulls toward LM-encoder geometry and away from contrastive geometry. The larger number was the reference row itself: CLIP-B/32 against T5-XXL read Jaccard .2598 / Spearman βˆ’.0016 β€” shared local neighbourhoods, zero global rank agreement. Minted as a constant, it was downgraded at 10:01 the same morning to an instance of the teacher-teacher disagreement axis registered on 08-18 (0.285); what it adds is that Spearman on cosines is invariant to a global rotation, so the disagreement is content, not frame.

(09:06.) PROBE-3c gauged every ruler against every other β€” T5-XXL, CLIP-L/14, CLIP-B/32, captionbert v1 and bert-base, each a final-layer pooled snapshot β€” and put the shop's representations against all five. Every cell is Jaccard@10 / Spearman on the one draw; the last two rows belong to the armed-encoder result below.

Jaccard@10 / Spearman t5_xxl clip_L clip_B32 captionbert v1 bert_base
ruler t5_xxl β€” .2739 / +.0597 .2598 / βˆ’.0016 .2412 / +.3476 .3170 / +.5982
ruler clip_L .2739 / +.0597 β€” .6495 / +.8784 .2596 / +.4937 .2618 / +.3167
ruler clip_B32 .2598 / βˆ’.0016 .6495 / +.8784 β€” .2537 / +.4526 .2485 / +.2504
ruler captionbert v1 .2412 / +.3476 .2596 / +.4937 .2537 / +.4526 β€” .3054 / +.6237
ruler bert_base .3170 / +.5982 .2618 / +.3167 .2485 / +.2504 .3054 / +.6237 β€”
blackboard L14 (2s, step 61,422) .3003 / +.4117 .3491 / +.4537 .3355 / +.3954 .3274 / +.6275 .3347 / +.6518
relay reads (wide-MoE arm) .2366 / +.3709 .2455 / +.2967 .2381 / +.2715 .2526 / +.5561 .2410 / +.5241
pooled, arm off (base surface) .2414 / +.2816 .2572 / +.2208 .2505 / +.2052 .2393 / +.3807 .2338 / +.3552
alephlm-0 a1-s0 (array representative) .3523 / +.4383 .3699 / +.5504 .3536 / +.4789 .4145 / +.7163 .5050 / +.7762
captionbert-v2-B, bare .3552 / +.4376 .3681 / +.5473 .3520 / +.4755 .4120 / +.7133 .5177 / +.7811
captionbert-v2-B, b-collective armed .3140 / +.2477 .4728 / +.6469 .4595 / +.5923 .3428 / +.5981 .3600 / +.5232

Two clusters: contrastive (the two CLIPs agree at +.878) and language-model (T5, bert, captionbert at +.348 to +.624). Between them only the T5 row is near zero (+.060 to clip_L, βˆ’.002 to B32); bert-base to clip_L reads +.317, a cell the journal's "globally unrelated" skips. Captionbert v1 is the best bridge in the battery (+.494 to clip_L, +.348 to T5), the 08-02 docket's partial answer. L14 sits in the shared core: it aligns with T5 (+.412) and with each CLIP (+.454, +.395) β€” roughly seven times the +.060 at which T5 aligns with clip_L, while T5 against B32 sits at zero (βˆ’.002) β€” and reads +.628 and +.652 against the LM rulers β€” though the blanket form "better than the clusters align with each other" fails one comparison, L14's +.395 on B32 against the +.494 bridge. The relay reads beat the pooled surface on Spearman on all five rulers, with a captionbert affinity of +.556, but lose neighbours to it on T5, clip_L and B32 β€” the journal states the rank half only. Pooled on-versus-off splits the same way: rank up on the three LM rulers (+.019 to +.023), down on both CLIPs (βˆ’.004, βˆ’.006), Jaccard down on all five (βˆ’.002 to βˆ’.008). None of these differences has a bar.

The expanded matrix: the distilled array

(09:16; Phil's steer.) The six alephlm-0 E1 finals (a1 anchored, a2 dense, a3 frozen-random addresses, two seeds each; 58.3M parameters β€” the mint's "26M" was corrected on 09-02) and captionbert-8192-v2 and v2-B bare joined the battery as pooled encoder outputs. The distilled array tops every external ruler: the six arms read +.436–.444 against T5, +.546–.556 against clip_L β€” never having trained against CLIP β€” +.474–.484 against B32, +.716–.720 against captionbert and +.776–.781 against bert, above L14 on all five. Two laws were extended: E1's amplitude-not-choice reaches conditioning geometry, and dedicated consensus distillation beats emergence at this size. The honest scope: the bert affinity is partly the objective, since the five-teacher consensus lives in the bert frame; the clip_L number is the real bridge evidence. And the band is wider than the journal's ".003 on every ruler": across the six arms it spans .0021–.0086 on Jaccard and .0035–.0097 on Spearman, so within .01 on every gauge is what the hub supports. v2 and v2-B bare land within .005 of the six-arm span on every ruler β€” inside it on clip_L and B32 on both gauges, within .001 on T5, .002–.004 below it on the captionbert ruler, and v2-B .004 above it on bert neighbours: the array is a lineage, not a routing effect.

PROBE-4: containment and binding

(09:35.) The gauge suite minted at 09:18 from a three-agent literature sweep β€” debiased CKA with a top-component drop, split-fit Procrustes scored held-out, bidirectional ten-fold ridge RΒ², then a functional binding probe β€” became PROBE-4a and 4b; its retro-fit found it had re-derived three house laws without the mandatory grep. Tier 3 on the a1-s0 arm, six-arm bands ≀ .008: T5 explains the array at RΒ² .776 Β± .007 and clip_L at .651 Β± .007 (the a2 cells .648); the array predicts T5 at .425 and clip_L at .590; Procrustes angle .88–.95 to T5 and the CLIPs. The array lives inside the rulers' span. L14 is the opposite character: it predicts clip_L at .600 Β± .009 and B32 at .625 Β± .008 β€” the best forward numbers in the battery on both β€” while T5 explains it at only .529 Β± .013, clip_L at .448 Β± .011, B32 at .432 Β± .011: surplus structure no ruler linearly explains. That reading is rank-confounded until a rank-matched control runs β€” L14 is a 32k-dimensional flattening β€” and the only effective ranks on these representations come from a different 1,250-row cache (bcd-e000: clip_L 249.5, L14 212.0, array 96.6, v2-B 101.4, armed 104.9). The wide-MoE reads' twenty cells (T5-contained .573, forward .320) were never verdicted.

The binding probe was a programmatic corpus of 300 scenes (attribute 113, relation 93, count 94), each with a multiset-preserving swap and a reworded anchor, plus 200 red-to-blue flips; the group score requires all four Winoground inequalities, nominal chance 25%. clip_L read accuracy .673 / group .143 / SNR .737 with attribute .566 (ARO's known CLIP weakness, reproduced) β€” the pooled-vector binding champion today. Pooled T5-XXL read .553 / .043 / .572: a swap moves it less than adding "a photo of" β€” functionally bag-of-words at the pooled interface, its binding living in token states. The distilled trio β€” captionbert v1 .533 / .000 / .129, the a2-s0 arm .537 / .000 / .148, v2-B bare .517 / .003 / .182 β€” is binding-dead when pooled; only one of six E1 arms was run, so the six-arm generalization rests on the tier-1/3 tie. L14 read .560 / .277 / .594 with attribute .708 and count .809, the best count binding in the battery and the only encoder above nominal chance β€” with the rider that three encoders scoring .000–.003 shows the calibration is off, and a measured random-encoder null is owed before that row travels as a rank. The relation column is retracted as a probe artifact: its anchors reverse mention order, so every order-sensitive encoder inverts (L14 .129) and clip_L's .742 reads as order-insensitivity rewarded. The reads and the base surface scored group .100 and .093 yet moved the most under swaps (SNR .785 and .899, the two highest); the journal's verdict that the reads do not survive this tier does not say which score it rests on. One column was computed and never cited: colour-flip coherence, where the binding-dead trio is the most coherent (.965–.967), T5 the least (.327), L14 between (.711) β€” the closest existing measurement of the factorization axis the doctrine still grades unmeasured.

The synthesis and its audit

As minted: three champions on three tiers β€” the array for drop-in containment, clip_L for pooled binding, L14 for core plus surplus β€” and "trust the highest tier" as the rule. The retro-audit at 10:01 reclassified it: cross-encoder maps are two-regime (btx-e001, 08-18), a memorizable lookup head and a compositional tail that never closes; containment gauges the head, binding the tail; the champions are one registered structure, tiers are never ranked against each other, and the trust clause is scoped to causal-conditioning claims. Binding-death is a target property: centroid-consensus pooled targets collapse spread by construction, mask-mean pooling is GAP-class, a multiset swap is an anagram probe β€” with the caveat that for aleph-family signed codes raw cosine reads the embedding of the code, not the code, so a sign-tier re-probe is owed. "Binding" throughout is compositional, not the registered 0.29154-rad constant; and the L14 core duplicates the Campaign-B alloy claim shape, which showed such a core can be grown β€” L14 is a growable object, not a found one.

The hub toggle and the armed encoder

(09:48; Phil: "did you test the hubs?") The alephlm-0 bed's own dispatch toggle (twelve banks, null path bit-exact) ran paired on two arms. a1-s0, learned addresses: the hub write rotates the output 49.0Β° (cosine .656) and carries both rulers' share β€” T5 +.4395 on β†’ +.3668 off, clip_L +.5521 β†’ +.4579. a3-s1, frozen-random addresses: 45.9Β° (cosine .695), zero T5 share (+.4413 β†’ +.4435) and the entire CLIP bridge (+.5553 β†’ +.3544, βˆ’.201). Binding stayed dead in all four states (accuracy .46–.54, group ≀ .003) while hub-off SNR doubled (.331 and .339 against .165 and .152). The refinement β€” endpoint invariance conceals arm-dependent division of labor: what is learned is partition-invariant, where it lives is address-dependent β€” carries four riders: the rotation is volume, not function (the per-ruler Spearman decomposition is the function number, and a3's collapse survives it); the two cells confound seed with arm, so a1-s1 and a3-s0 are owed, still queued on 09-04; "first separation" is scoped to these arms; and it fuses with the battery night's B2 as one shape β€” causal location is the free variable.

(11:04; Phil caught that every probe had used the bare trunk.) captionbert-v2-B with its native b-collective attached (the .6031 β†’ .7294 eight-task STS configuration) rotates its embedding by .5525 on 256 captions and migrates cluster on 5/5 rulers, the last two rows of the table: T5 +.4376 β†’ +.2477, clip_L +.5473 β†’ +.6469 (the best house CLIP alignment, above every E1 arm), B32 +.4755 β†’ +.5923, captionbert +.7133 β†’ +.5981, bert +.7811 β†’ +.5232. The journal's printed "bare" column carries array-representative values on three rulers (+.551, +.478, +.717) rather than these hub cells, and the manifest still reads "+.547 β†’ +.248" for T5 after the 09-02 correction to +.438 β†’ +.248; the deltas match either way. Arm state selects the conditioning cluster β€” one artifact serving CLIP-class targets with arms on and T5-class with them off β€” is a candidate on one draw and one arm-set, its second collective queued. Binding is dead in both states (.490 / .010 / .194 armed, .517 / .003 / .182 bare, the 4b row reproduced exactly): a +.126 task gain buys zero composition. A tooling rider cost two rows before the catch: the attach handle's all_off() is a context manager; called bare it is a silent no-op, its signature a delta of exactly 0.

What the record lacks

(10:01 and after.) One factual correction: PROBE-3c's line that the 08-02 docket's four token-state probes "remain open" was false β€” they ran on 08-04 (captionbert-B certified at prompts of 256 tokens or fewer), and PROBE-4b extends that record. But the audit's own second clause, that pooled-T5 bag-of-words was "already measured then," is not supported by the 08-04 ledger, which gauged token-position binding on t5-base with no pooled swap and no XXL: not in the record beats the claim. What survives as law is convergent with B1 and B4 of section 6 β€” head-side attachments cannot reach mid-stack structure: 90% of L14 under T5, group .10 against .277 β€” and its falsifier, a governed constellation attached at layer 14, is the flagship rung. Rung 0 β€” null and shuffled-pairing columns, effective ranks, a rank-matched L14 control, a second caption draw with a same-draw repeat, the relation-anchor fix β€” was minted as mandatory before any number travels further and is item 1 of the live queue on 09-04, unrun; the four-cell hub deconfound is item 2, the second collective item 4. Rungs 2–5 (token-state, sign-tier, ruler-position, depth-by-family, frame-alloy, L14-attach) are unrun in their own form; what ran instead, the same evening, was the bcd-e campaign of section 9 β€” the only test any of these representations has had on a generator bed.