Create prototype_5_output.txt

prototype_5_output.txt

@@ -0,0 +1,225 @@
+=================================================================
+RAPID PROTOTYPE v2: Differentiation-Centered Bank
+=================================================================
+  Device: cuda
+
+=================================================================
+PHASE 0: EXTRACTION
+=================================================================
+  Captions: 20,000
+
+  Extracting: bert...
+Loading weights: 100%
+ 199/199 [00:00<00:00, 4038.86it/s, Materializing param=pooler.dense.weight]
+BertModel LOAD REPORT from: google-bert/bert-base-uncased
+Key                                        | Status     |  | 
+-------------------------------------------+------------+--+-
+cls.predictions.bias                       | UNEXPECTED |  | 
+cls.predictions.transform.dense.bias       | UNEXPECTED |  | 
+cls.predictions.transform.LayerNorm.weight | UNEXPECTED |  | 
+cls.predictions.transform.LayerNorm.bias   | UNEXPECTED |  | 
+cls.seq_relationship.weight                | UNEXPECTED |  | 
+cls.predictions.transform.dense.weight     | UNEXPECTED |  | 
+cls.seq_relationship.bias                  | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    bert: 100%|██████████| 157/157 [00:23<00:00,  6.55it/s]
+    Shape: torch.Size([20000, 768])
+
+  Extracting: modern...
+Loading weights: 100%
+ 134/134 [00:00<00:00, 4016.84it/s, Materializing param=layers.21.mlp_norm.weight]
+ModernBertModel LOAD REPORT from: answerdotai/ModernBERT-base
+Key               | Status     |  | 
+------------------+------------+--+-
+head.dense.weight | UNEXPECTED |  | 
+head.norm.weight  | UNEXPECTED |  | 
+decoder.bias      | UNEXPECTED |  | 
+
+Notes:
+- UNEXPECTED	:can be ignored when loading from different task/architecture; not ok if you expect identical arch.
+    modern: 100%|██████████| 157/157 [00:35<00:00,  4.39it/s]
+    Shape: torch.Size([20000, 768])
+
+=================================================================
+PHASE 0b: GENERALIZED PROCRUSTES ALIGNMENT (no reference bias)
+=================================================================
+  GPA iter 1: delta=1.19668072
+  GPA iter 3: delta=0.00029225
+  GPA iter 6: delta=0.00006347
+  GPA iter 9: delta=0.00002718
+  bert      : cos_after=0.8541  cos_to_mean=0.9865
+  modern    : cos_after=0.8577  cos_to_mean=0.9867
+  cos(consensus, bert): 0.9867
+  cos(consensus, modern): 0.9868
+  Equidistance range: 0.0001 (should be near 0)
+
+  Measuring consensus statistics...
+    CV:       0.1771
+    Mean cos: 0.0018
+    Eff dim:  109.5
+    Spectral: [0.0343, 0.0322, 0.0275, 0.0240, 0.0222...]
+
+=================================================================
+PHASE 1: TRAIN STUDENT
+=================================================================
+  Student: 11,269,632 params
+  CV target: 0.1771
+  E1: 2s  loss=2.9588  t_acc=0.362  t_cos=0.334  v_acc=0.494  v_cos=0.503  v_cv=0.223
+  E2: 2s  loss=1.4268  t_acc=0.761  t_cos=0.543  v_acc=0.704  v_cos=0.588  v_cv=0.212
+  E3: 2s  loss=0.9784  t_acc=0.887  t_cos=0.604  v_acc=0.822  v_cos=0.639  v_cv=0.182
+  E4: 2s  loss=0.7289  t_acc=0.943  t_cos=0.641  v_acc=0.912  v_cos=0.676  v_cv=0.182
+  E5: 2s  loss=0.5807  t_acc=0.968  t_cos=0.666  v_acc=0.920  v_cos=0.686  v_cv=0.182
+
+  Student saved. v_cos=0.686, v_cv=0.182
+
+=================================================================
+PHASE 2: TRAIN ALIGNMENT BANK (student frozen)
+=================================================================
+  Pre-encoding through frozen student...
+  Student embeddings: torch.Size([18000, 768])
+    Expert 0 (bert): rotation + whitener + mean loaded, cos_after=0.8541
+    Expert 1 (modern): rotation + whitener + mean loaded, cos_after=0.8577
+    Anchors: 512 initialized from consensus embeddings
+    Targets: CV=0.1771, mean_cos=0.0018
+  Bank: 2,921,088 params
+  Bank targets: CV=0.1771, mean_cos=0.0018
+    Calibrated disagreement:
+      cross_cos: 0.0794 ± 0.0035
+      disagree_ratio: 0.000000
+
+  E 1: 1s  loss=0.4775  v_loss=0.4350
+    Geometry:  b_cv=0.2673  e_cv=0.1680  spread=0.03938  a_max=0.653
+    Experts:   cos=0.791±0.006  agr=0.000142  ortho=0.000440
+    Disagree:  x_cos=0.0833±0.0019  ratio=0.004748  preserve=0.014587  norms=0.1814
+
+  E 2: 1s  loss=0.3981  v_loss=0.3833
+    Geometry:  b_cv=0.2230  e_cv=0.1651  spread=0.02783  a_max=0.669
+    Experts:   cos=0.809±0.006  agr=0.000008  ortho=0.000344
+    Disagree:  x_cos=0.0817±0.0018  ratio=0.003509  preserve=0.000024  norms=0.1719
+
+  E 3: 1s  loss=0.3730  v_loss=0.3757
+    Geometry:  b_cv=0.2162  e_cv=0.1648  spread=0.02493  a_max=0.670
+    Experts:   cos=0.830±0.005  agr=0.000004  ortho=0.000186
+    Disagree:  x_cos=0.0799±0.0019  ratio=0.002291  preserve=0.000013  norms=0.1513
+
+  E 4: 1s  loss=0.3623  v_loss=0.3708
+    Geometry:  b_cv=0.2187  e_cv=0.1615  spread=0.02314  a_max=0.670
+    Experts:   cos=0.832±0.005  agr=0.000003  ortho=0.000115
+    Disagree:  x_cos=0.0793��0.0020  ratio=0.003285  preserve=0.000011  norms=0.1422
+
+  E 5: 1s  loss=0.3554  v_loss=0.3539
+    Geometry:  b_cv=0.2141  e_cv=0.1621  spread=0.02139  a_max=0.669
+    Experts:   cos=0.853±0.004  agr=0.000002  ortho=0.000079
+    Disagree:  x_cos=0.0781±0.0021  ratio=0.001270  preserve=0.000011  norms=0.0980
+
+  E 6: 1s  loss=0.3507  v_loss=0.3571
+    Geometry:  b_cv=0.2124  e_cv=0.1633  spread=0.02019  a_max=0.669
+    Experts:   cos=0.829±0.005  agr=0.000001  ortho=0.000058
+    Disagree:  x_cos=0.0788±0.0022  ratio=0.001736  preserve=0.000010  norms=0.1789
+
+  E 7: 1s  loss=0.3460  v_loss=0.3465
+    Geometry:  b_cv=0.2059  e_cv=0.1607  spread=0.01903  a_max=0.669
+    Experts:   cos=0.845±0.005  agr=0.000001  ortho=0.000045
+    Disagree:  x_cos=0.0819±0.0023  ratio=0.001425  preserve=0.000008  norms=0.1536
+
+  E 8: 1s  loss=0.3449  v_loss=0.3421
+    Geometry:  b_cv=0.2060  e_cv=0.1592  spread=0.01841  a_max=0.670
+    Experts:   cos=0.833±0.005  agr=0.000003  ortho=0.000035
+    Disagree:  x_cos=0.0885±0.0021  ratio=0.001539  preserve=0.000017  norms=0.1313
+
+  E 9: 1s  loss=0.3422  v_loss=0.3451
+    Geometry:  b_cv=0.2040  e_cv=0.1626  spread=0.01793  a_max=0.669
+    Experts:   cos=0.822±0.005  agr=0.000003  ortho=0.000031
+    Disagree:  x_cos=0.0761±0.0024  ratio=0.001610  preserve=0.000037  norms=0.2032
+
+  E10: 1s  loss=0.3416  v_loss=0.3497
+    Geometry:  b_cv=0.2077  e_cv=0.1647  spread=0.01735  a_max=0.669
+    Experts:   cos=0.782±0.007  agr=0.000003  ortho=0.000029
+    Disagree:  x_cos=0.0825±0.0023  ratio=0.004691  preserve=0.000025  norms=0.2039
+
+  E11: 1s  loss=0.3387  v_loss=0.3507
+    Geometry:  b_cv=0.2019  e_cv=0.1640  spread=0.01701  a_max=0.668
+    Experts:   cos=0.811±0.005  agr=0.000002  ortho=0.000024
+    Disagree:  x_cos=0.0780±0.0023  ratio=0.000957  preserve=0.000015  norms=0.1889
+
+  E12: 1s  loss=0.3391  v_loss=0.3381
+    Geometry:  b_cv=0.2006  e_cv=0.1588  spread=0.01675  a_max=0.668
+    Experts:   cos=0.778±0.006  agr=0.000003  ortho=0.000021
+    Disagree:  x_cos=0.0729±0.0021  ratio=0.001148  preserve=0.000024  norms=0.1404
+
+  E13: 1s  loss=0.3373  v_loss=0.3434
+    Geometry:  b_cv=0.1987  e_cv=0.1635  spread=0.01671  a_max=0.668
+    Experts:   cos=0.703±0.007  agr=0.000013  ortho=0.000021
+    Disagree:  x_cos=0.0680±0.0026  ratio=0.003978  preserve=0.000085  norms=0.2265
+
+  E14: 1s  loss=0.3383  v_loss=0.3351
+    Geometry:  b_cv=0.2027  e_cv=0.1658  spread=0.01634  a_max=0.668
+    Experts:   cos=0.779±0.005  agr=0.000007  ortho=0.000024
+    Disagree:  x_cos=0.0849±0.0022  ratio=0.002337  preserve=0.000085  norms=0.1472
+
+  E15: 1s  loss=0.3366  v_loss=0.3357
+    Geometry:  b_cv=0.1999  e_cv=0.1612  spread=0.01584  a_max=0.668
+    Experts:   cos=0.671±0.008  agr=0.000008  ortho=0.000023
+    Disagree:  x_cos=0.0777±0.0024  ratio=0.011179  preserve=0.000061  norms=0.1758
+
+  E16: 1s  loss=0.3363  v_loss=0.3467
+    Geometry:  b_cv=0.1983  e_cv=0.1612  spread=0.01575  a_max=0.668
+    Experts:   cos=0.737±0.005  agr=0.000010  ortho=0.000022
+    Disagree:  x_cos=0.0839±0.0022  ratio=0.006047  preserve=0.000049  norms=0.1216
+
+  E17: 1s  loss=0.3343  v_loss=0.3376
+    Geometry:  b_cv=0.1974  e_cv=0.1655  spread=0.01591  a_max=0.668
+    Experts:   cos=0.718±0.005  agr=0.000002  ortho=0.000020
+    Disagree:  x_cos=0.0723±0.0023  ratio=0.002539  preserve=0.000042  norms=0.0947
+
+  E18: 1s  loss=0.3354  v_loss=0.3457
+    Geometry:  b_cv=0.1955  e_cv=0.1580  spread=0.01588  a_max=0.668
+    Experts:   cos=0.763±0.005  agr=0.000007  ortho=0.000019
+    Disagree:  x_cos=0.0796±0.0022  ratio=0.004057  preserve=0.000069  norms=0.1001
+
+  E19: 1s  loss=0.3344  v_loss=0.3313
+    Geometry:  b_cv=0.1962  e_cv=0.1602  spread=0.01560  a_max=0.668
+    Experts:   cos=0.687±0.005  agr=0.000005  ortho=0.000018
+    Disagree:  x_cos=0.0862±0.0024  ratio=0.005997  preserve=0.000030  norms=0.1218
+
+  E20: 1s  loss=0.3331  v_loss=0.3651
+    Geometry:  b_cv=0.1950  e_cv=0.1631  spread=0.01556  a_max=0.668
+    Experts:   cos=0.729±0.005  agr=0.000007  ortho=0.000018
+    Disagree:  x_cos=0.0826±0.0021  ratio=0.006963  preserve=0.000065  norms=0.0781
+
+=================================================================
+PHASE 3: GEOMETRIC VERIFICATION
+=================================================================
+  Passthrough:     1.000000 (target: 1.000)
+  Emb CV:          0.1660 (consensus: 0.1771)
+  Geo context CV:  0.2053
+  Geo eff_dim:     30.5 / 128
+  Expert cos:      0.729 ± 0.005
+  Anchor max cos:  0.668
+  Disagreement:
+    Cross-expert:  0.0826 ± 0.0021
+    Ratio:         0.006963 (target: 0.000000)
+    Norm spread:   0.0781
+
+=================================================================
+PHASE 4: CLASSIFIER STABILITY TEST
+=================================================================
+  with_bank      : train=0.746  val=0.500  gap=0.246
+  without_bank   : train=0.490  val=0.363  gap=0.126
+
+=================================================================
+SUMMARY
+=================================================================
+  Consensus CV:     0.1771
+  Consensus eff_dim:109.5
+  Student v_cos:    0.686
+  Student v_cv:     0.182
+  Bank params:      2,921,088
+  Bank geo_eff_dim: 30.5
+  Bank geo_cv:      0.2053
+
+=================================================================
+DONE
+=================================================================