Create prototype_55_output.txt

prototype_55_output.txt

@@ -0,0 +1,232 @@
+=================================================================
+RAPID PROTOTYPE v2: Differentiation-Centered Bank
+=================================================================
+  Device: cuda
+
+=================================================================
+PHASE 0: EXTRACTION
+=================================================================
+  Captions: 20,000
+
+  Extracting: bert...
+Loading weights: 100%
+ 199/199 [00:00<00:00, 4216.36it/s, Materializing param=pooler.dense.weight]
+BertModel LOAD REPORT from: google-bert/bert-base-uncased
+Key                                        | Status     |  | 
+-------------------------------------------+------------+--+-
+cls.predictions.bias                       | UNEXPECTED |  | 
+cls.predictions.transform.dense.bias       | UNEXPECTED |  | 
+cls.predictions.transform.LayerNorm.weight | UNEXPECTED |  | 
+cls.predictions.transform.LayerNorm.bias   | UNEXPECTED |  | 
+cls.seq_relationship.weight                | UNEXPECTED |  | 
+cls.predictions.transform.dense.weight     | UNEXPECTED |  | 
+cls.seq_relationship.bias                  | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    bert: 100%|██████████| 157/157 [00:23<00:00,  6.56it/s]
+    Shape: torch.Size([20000, 768])
+
+  Extracting: modern...
+Loading weights: 100%
+ 134/134 [00:00<00:00, 4047.07it/s, Materializing param=layers.21.mlp_norm.weight]
+ModernBertModel LOAD REPORT from: answerdotai/ModernBERT-base
+Key               | Status     |  | 
+------------------+------------+--+-
+head.dense.weight | UNEXPECTED |  | 
+head.norm.weight  | UNEXPECTED |  | 
+decoder.bias      | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    modern: 100%|██████████| 157/157 [00:35<00:00,  4.39it/s]
+    Shape: torch.Size([20000, 768])
+
+=================================================================
+PHASE 0b: GENERALIZED PROCRUSTES ALIGNMENT (no reference bias)
+=================================================================
+  GPA iter 1: delta=1.19668072
+  GPA iter 3: delta=0.00029225
+  GPA iter 6: delta=0.00006347
+  GPA iter 9: delta=0.00002718
+  bert      : cos_after=0.8541  cos_to_mean=0.9865
+  modern    : cos_after=0.8577  cos_to_mean=0.9867
+  cos(consensus, bert): 0.9867
+  cos(consensus, modern): 0.9868
+  Equidistance range: 0.0001 (should be near 0)
+
+  Measuring consensus statistics...
+    CV:       0.1771
+    Mean cos: 0.0018
+    Eff dim:  109.5
+    Spectral: [0.0343, 0.0322, 0.0275, 0.0240, 0.0222...]
+
+=================================================================
+PHASE 1: TRAIN STUDENT
+=================================================================
+  Student: 11,269,632 params
+  CV target: 0.1771
+  E1: 2s  loss=2.9588  t_acc=0.362  t_cos=0.334  v_acc=0.494  v_cos=0.503  v_cv=0.223
+  E2: 2s  loss=1.4268  t_acc=0.761  t_cos=0.543  v_acc=0.704  v_cos=0.588  v_cv=0.212
+  E3: 2s  loss=0.9784  t_acc=0.887  t_cos=0.604  v_acc=0.822  v_cos=0.639  v_cv=0.182
+  E4: 2s  loss=0.7289  t_acc=0.943  t_cos=0.641  v_acc=0.912  v_cos=0.676  v_cv=0.182
+  E5: 2s  loss=0.5807  t_acc=0.968  t_cos=0.666  v_acc=0.920  v_cos=0.686  v_cv=0.182
+
+  Student saved. v_cos=0.686, v_cv=0.182
+
+=================================================================
+PHASE 2: TRAIN ALIGNMENT BANK (student frozen)
+=================================================================
+  Pre-encoding through frozen student...
+  Student embeddings: torch.Size([18000, 768])
+    Expert 0 (bert): rotation + whitener + mean loaded, cos_after=0.8541
+    Expert 1 (modern): rotation + whitener + mean loaded, cos_after=0.8577
+    Anchors: 512 initialized from consensus embeddings
+    Targets: CV=0.1771, mean_cos=0.0018
+  Bank: 2,921,088 params
+  Bank targets: CV=0.1771, mean_cos=0.0018
+    Calibrated disagreement (n=2000):
+      cross_cos: 0.0794 ± 0.0035
+      disagree_ratio: median=0.000000  mean=0.000000  std=0.000000
+      expert_cos: 1.0000 ± 0.0000
+
+  E 1: 1s  loss=0.4789  v_loss=0.4172
+    Geometry:  b_cv=0.2688  e_cv=0.1603  spread=0.03940  a_max=0.652
+    Experts:   cos=0.794±0.006  agr=0.000092  ortho=0.000388
+    Disagree:  x_cos=0.0740±0.0009  ratio=0.004326  preserve=0.013135  norms=0.1626
+
+  E 2: 1s  loss=0.4002  v_loss=0.3818
+    Geometry:  b_cv=0.2229  e_cv=0.1588  spread=0.02779  a_max=0.668
+    Experts:   cos=0.807±0.006  agr=0.000007  ortho=0.000288
+    Disagree:  x_cos=0.0805±0.0014  ratio=0.003575  preserve=0.000024  norms=0.1703
+
+  E 3: 1s  loss=0.3743  v_loss=0.3625
+    Geometry:  b_cv=0.2189  e_cv=0.1606  spread=0.02500  a_max=0.670
+    Experts:   cos=0.835±0.005  agr=0.000005  ortho=0.000152
+    Disagree:  x_cos=0.0774±0.0018  ratio=0.002279  preserve=0.000016  norms=0.1066
+
+  E 4: 1s  loss=0.3591  v_loss=0.3615
+    Geometry:  b_cv=0.2100  e_cv=0.1643  spread=0.02302  a_max=0.670
+    Experts:   cos=0.822±0.005  agr=0.000003  ortho=0.000094
+    Disagree:  x_cos=0.0781±0.0021  ratio=0.001569  preserve=0.000020  norms=0.1137
+
+  E 5: 1s  loss=0.3537  v_loss=0.3665
+    Geometry:  b_cv=0.2118  e_cv=0.1664  spread=0.02133  a_max=0.670
+    Experts:   cos=0.815±0.006  agr=0.000002  ortho=0.000066
+    Disagree:  x_cos=0.0765±0.0021  ratio=0.001389  preserve=0.000026  norms=0.1669
+
+  E 6: 1s  loss=0.3506  v_loss=0.3527
+    Geometry:  b_cv=0.2097  e_cv=0.1600  spread=0.02009  a_max=0.670
+    Experts:   cos=0.829±0.005  agr=0.000003  ortho=0.000048
+    Disagree:  x_cos=0.0846±0.0024  ratio=0.001772  preserve=0.000021  norms=0.1363
+
+  E 7: 1s  loss=0.3459  v_loss=0.3502
+    Geometry:  b_cv=0.2055  e_cv=0.1628  spread=0.01906  a_max=0.670
+    Experts:   cos=0.759±0.007  agr=0.000004  ortho=0.000040
+    Disagree:  x_cos=0.0774±0.0022  ratio=0.003070  preserve=0.000049  norms=0.1964
+
+  E 8: 1s  loss=0.3442  v_loss=0.3479
+    Geometry:  b_cv=0.2078  e_cv=0.1643  spread=0.01817  a_max=0.669
+    Experts:   cos=0.745±0.007  agr=0.000003  ortho=0.000033
+    Disagree:  x_cos=0.0782±0.0023  ratio=0.001258  preserve=0.000021  norms=0.1772
+
+  E 9: 1s  loss=0.3419  v_loss=0.3451
+    Geometry:  b_cv=0.2015  e_cv=0.1646  spread=0.01756  a_max=0.670
+    Experts:   cos=0.767±0.006  agr=0.000007  ortho=0.000030
+    Disagree:  x_cos=0.0823±0.0024  ratio=0.001625  preserve=0.000049  norms=0.2007
+
+  E10: 1s  loss=0.3433  v_loss=0.3433
+    Geometry:  b_cv=0.2074  e_cv=0.1594  spread=0.01746  a_max=0.669
+    Experts:   cos=0.762±0.005  agr=0.000006  ortho=0.000026
+    Disagree:  x_cos=0.0766±0.0018  ratio=0.001418  preserve=0.000073  norms=0.0529
+
+  E11: 1s  loss=0.3392  v_loss=0.3501
+    Geometry:  b_cv=0.2021  e_cv=0.1609  spread=0.01705  a_max=0.669
+    Experts:   cos=0.721±0.007  agr=0.000004  ortho=0.000026
+    Disagree:  x_cos=0.0698±0.0022  ratio=0.006405  preserve=0.000037  norms=0.1509
+
+  E12: 1s  loss=0.3383  v_loss=0.3534
+    Geometry:  b_cv=0.1983  e_cv=0.1639  spread=0.01693  a_max=0.668
+    Experts:   cos=0.753±0.005  agr=0.000014  ortho=0.000026
+    Disagree:  x_cos=0.0743±0.0021  ratio=0.000903  preserve=0.000076  norms=0.0763
+
+  E13: 1s  loss=0.3374  v_loss=0.3398
+    Geometry:  b_cv=0.1996  e_cv=0.1603  spread=0.01660  a_max=0.669
+    Experts:   cos=0.714±0.006  agr=0.000004  ortho=0.000022
+    Disagree:  x_cos=0.0791±0.0021  ratio=0.006335  preserve=0.000060  norms=0.1257
+
+  E14: 1s  loss=0.3376  v_loss=0.3415
+    Geometry:  b_cv=0.1992  e_cv=0.1657  spread=0.01647  a_max=0.669
+    Experts:   cos=0.704±0.006  agr=0.000006  ortho=0.000022
+    Disagree:  x_cos=0.0824±0.0021  ratio=0.006577  preserve=0.000061  norms=0.0873
+
+  E15: 1s  loss=0.3372  v_loss=0.3409
+    Geometry:  b_cv=0.2003  e_cv=0.1615  spread=0.01635  a_max=0.669
+    Experts:   cos=0.745±0.005  agr=0.000003  ortho=0.000019
+    Disagree:  x_cos=0.0760±0.0020  ratio=0.002660  preserve=0.000045  norms=0.0958
+
+  E16: 1s  loss=0.3355  v_loss=0.3328
+    Geometry:  b_cv=0.1990  e_cv=0.1601  spread=0.01600  a_max=0.669
+    Experts:   cos=0.689±0.005  agr=0.000004  ortho=0.000018
+    Disagree:  x_cos=0.0814±0.0024  ratio=0.002029  preserve=0.000042  norms=0.1414
+
+  E17: 1s  loss=0.3350  v_loss=0.3432
+    Geometry:  b_cv=0.1945  e_cv=0.1604  spread=0.01603  a_max=0.668
+    Experts:   cos=0.751±0.003  agr=0.000028  ortho=0.000020
+    Disagree:  x_cos=0.0825±0.0023  ratio=0.001129  preserve=0.000155  norms=0.0187
+
+  E18: 1s  loss=0.3372  v_loss=0.3336
+    Geometry:  b_cv=0.2044  e_cv=0.1605  spread=0.01590  a_max=0.668
+    Experts:   cos=0.720±0.003  agr=0.000004  ortho=0.000022
+    Disagree:  x_cos=0.0799±0.0020  ratio=0.002103  preserve=0.000055  norms=0.0331
+
+  E19: 1s  loss=0.3326  v_loss=0.3456
+    Geometry:  b_cv=0.1948  e_cv=0.1654  spread=0.01562  a_max=0.668
+    Experts:   cos=0.741±0.003  agr=0.000004  ortho=0.000021
+    Disagree:  x_cos=0.0797±0.0019  ratio=0.003153  preserve=0.000054  norms=0.0169
+
+  E20: 1s  loss=0.3351  v_loss=0.3460
+    Geometry:  b_cv=0.1992  e_cv=0.1596  spread=0.01567  a_max=0.668
+    Experts:   cos=0.725±0.005  agr=0.000002  ortho=0.000018
+    Disagree:  x_cos=0.0776±0.0023  ratio=0.008188  preserve=0.000053  norms=0.0326
+
+=================================================================
+PHASE 3: GEOMETRIC VERIFICATION
+=================================================================
+  Passthrough:     1.000000 (target: 1.000)
+  Emb CV:          0.1635 (consensus: 0.1771)
+  Geo context CV:  0.1892
+  Geo eff_dim:     30.7 / 128
+  Expert cos:      0.725 ± 0.005
+  Anchor max cos:  0.668
+  Disagreement:
+    Cross-expert:  0.0776 ± 0.0023
+    Ratio:         0.008188 (target: 0.000000)
+    Norm spread:   0.0326
+
+=================================================================
+PHASE 4: CLASSIFIER STABILITY TEST
+=================================================================
+
+  Mode                    Dim   Train     Val     Gap
+  --------------------------------------------------
+  raw_768                1536   0.498   0.357   0.141
+  raw+diff               3072   0.567   0.475   0.092
+  bank_enriched          1792   0.766   0.532   0.235
+  bank+diff              3584   0.722   0.670   0.052
+  geo_explicit              6   0.326   0.363  -0.037
+
+=================================================================
+SUMMARY
+=================================================================
+  Consensus CV:     0.1771
+  Consensus eff_dim:109.5
+  Student v_cos:    0.686
+  Student v_cv:     0.182
+  Bank params:      2,921,088
+  Bank geo_eff_dim: 30.7
+  Bank geo_cv:      0.1892
+
+=================================================================
+DONE
+=================================================================