| | --- |
| | license: apache-2.0 |
| | --- |
| | |
| | # Partial Success: Prototype 5.5 |
| |
|
| | Full geodesic alignment possible at a cost of additional dimensions per geodesic complexity grade applied. |
| |
|
| | I'm thinking the losses have too many changes, and the system is drifting from the directed goals specified in the earlier versions. |
| | There many be too many differences between the losses here, the process here, and the system here - in conjunction with the geolip structural integrity. |
| | This process may be useful for head training, but it may not be useful for distillation training. |
| |
|
| |
|
| | ``` |
| | Mode Dim Train Val Gap |
| | -------------------------------------------------- |
| | raw_768 1536 0.498 0.357 0.141 |
| | raw+diff 3072 0.567 0.475 0.092 |
| | bank_enriched 1792 0.766 0.532 0.235 |
| | bank+diff 3584 0.722 0.670 0.052 |
| | geo_explicit 6 0.326 0.363 -0.037 |
| | |
| | ``` |
| |
|
| | # prototype 1; |
| | full anchor bank, minimal anchors, positive outcome from mini distilled expert student |
| |
|
| | bert-base aligned. |
| |
|
| | Positive enough to continue refinement. |
| |
|
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough integrity: 1.000000 (should be ~1.000) |
| | Geo context CV: 0.1651 |
| | Geo context eff_dim: 21.5 |
| | Geo context shape: torch.Size([2000, 64]) |
| | |
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | with_bank : train_acc=0.481 val_acc=0.390 gap=0.091 |
| | without_bank : train_acc=0.443 val_acc=0.330 gap=0.113 |
| | |
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | |
| | Student: mini_student.pt |
| | Bank: alignment_bank.pt |
| | Consensus CV: 0.1316 |
| | Student v_cos: 0.612 |
| | ``` |
| | |
| | # prototype 2; |
| | full anchor bank self aligned, 512 anchors, reduced training overfit, increased validation on distilled expert student |
| | |
| | bert-base aligned. |
| | |
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough: 1.000000 (target: 1.000) |
| | Emb CV: 0.1592 (consensus: 0.1316) |
| | Geo context CV: 0.1790 |
| | Geo eff_dim: 33.7 / 128 |
| | Expert cos: 0.794 ± 0.052 |
| | Anchor max cos: 0.656 |
| | Cross-expert: 0.032 |
| | |
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | with_bank : train=0.782 val=0.505 gap=0.277 |
| | without_bank : train=0.512 val=0.372 gap=0.140 |
| | |
| | ================================================================= |
| | SUMMARY |
| | ================================================================= |
| | Consensus CV: 0.1316 |
| | Consensus eff_dim:223.7 |
| | Student v_cos: 0.612 |
| | Student v_cv: 0.169 |
| | Bank params: 2,918,784 |
| | Bank geo_eff_dim: 33.7 |
| | Bank geo_cv: 0.1790 |
| | |
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | ``` |
| | |
| | # prototype 3; |
| | full anchor bank, 512 anchors, full procrustes whitening, dynamic ortho, full geometric statistics, even more improved accuracy |
| | |
| | dynamic alignment, additional utility potential present from dynamic capacity than directly aligning to an expert. |
| | |
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough: 1.000000 (target: 1.000) |
| | Emb CV: 0.1592 (consensus: 0.1316) |
| | Geo context CV: 0.1817 |
| | Geo eff_dim: 33.5 / 128 |
| | Expert cos: 0.800 ± 0.004 |
| | Anchor max cos: 0.656 |
| | Cross-expert: -0.107 |
| |
|
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | with_bank : train=0.773 val=0.513 gap=0.260 |
| | without_bank : train=0.512 val=0.372 gap=0.140 |
| |
|
| | ================================================================= |
| | SUMMARY |
| | ================================================================= |
| | Consensus CV: 0.1316 |
| | Consensus eff_dim:223.7 |
| | Student v_cos: 0.612 |
| | Student v_cv: 0.169 |
| | Bank params: 2,920,320 |
| | Bank geo_eff_dim: 33.5 |
| | Bank geo_cv: 0.1817 |
| |
|
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | ``` |
| | |
| | # prototype 4; |
| | full anchor bank, 512 anchors, full procrestes whitening, dynamic ortho, improved and robust relative geometric statistics, even more improved accuracy |
| | |
| | with full dynamic alignment the system is now cross-expert positive with a dynamic series of ratios for loss and alignment |
| | |
| | stability still present, additional complexity can be included. |
| | |
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough: 1.000000 (target: 1.000) |
| | Emb CV: 0.1557 (consensus: 0.1316) |
| | Geo context CV: 0.1872 |
| | Geo eff_dim: 33.2 / 128 |
| | Expert cos: 0.784 ± 0.005 |
| | Anchor max cos: 0.655 |
| | Disagreement: |
| | Cross-expert: 0.0463 ± 0.0154 |
| | Ratio: 0.001912 (target: 0.000000) |
| | Norm spread: 0.1279 |
| | |
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | with_bank : train=0.775 val=0.508 gap=0.266 |
| | without_bank : train=0.513 val=0.392 gap=0.122 |
| | |
| | ================================================================= |
| | SUMMARY |
| | ================================================================= |
| | Consensus CV: 0.1316 |
| | Consensus eff_dim:223.7 |
| | Student v_cos: 0.612 |
| | Student v_cv: 0.169 |
| | Bank params: 2,921,088 |
| | Bank geo_eff_dim: 33.2 |
| | Bank geo_cv: 0.1872 |
| | |
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | ``` |
| | |
| | # prototype 5 |
| | |
| | Attempted equidistance geodesic alignment protocol. Successful, perfect harmony to reach 0.2 context CV ratio possible. |
| | |
| | |
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough: 1.000000 (target: 1.000) |
| | Emb CV: 0.1660 (consensus: 0.1771) |
| | Geo context CV: 0.2053 |
| | Geo eff_dim: 30.5 / 128 |
| | Expert cos: 0.729 ± 0.005 |
| | Anchor max cos: 0.668 |
| | Disagreement: |
| | Cross-expert: 0.0826 ± 0.0021 |
| | Ratio: 0.006963 (target: 0.000000) |
| | Norm spread: 0.0781 |
| | |
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | with_bank : train=0.746 val=0.500 gap=0.246 |
| | without_bank : train=0.490 val=0.363 gap=0.126 |
| |
|
| | ================================================================= |
| | SUMMARY |
| | ================================================================= |
| | Consensus CV: 0.1771 |
| | Consensus eff_dim:109.5 |
| | Student v_cos: 0.686 |
| | Student v_cv: 0.182 |
| | Bank params: 2,921,088 |
| | Bank geo_eff_dim: 30.5 |
| | Bank geo_cv: 0.2053 |
| |
|
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | ``` |
| | |
| | # prototype 5.5 correct geodesic relational contextualization |
| | |
| | |
| | ``` |
| | ================================================================= |
| | PHASE 3: GEOMETRIC VERIFICATION |
| | ================================================================= |
| | Passthrough: 1.000000 (target: 1.000) |
| | Emb CV: 0.1635 (consensus: 0.1771) |
| | Geo context CV: 0.1892 |
| | Geo eff_dim: 30.7 / 128 |
| | Expert cos: 0.725 ± 0.005 |
| | Anchor max cos: 0.668 |
| | Disagreement: |
| | Cross-expert: 0.0776 ± 0.0023 |
| | Ratio: 0.008188 (target: 0.000000) |
| | Norm spread: 0.0326 |
| | |
| | ================================================================= |
| | PHASE 4: CLASSIFIER STABILITY TEST |
| | ================================================================= |
| | |
| | Mode Dim Train Val Gap |
| | -------------------------------------------------- |
| | raw_768 1536 0.498 0.357 0.141 |
| | raw+diff 3072 0.567 0.475 0.092 |
| | bank_enriched 1792 0.766 0.532 0.235 |
| | bank+diff 3584 0.722 0.670 0.052 |
| | geo_explicit 6 0.326 0.363 -0.037 |
| |
|
| | ================================================================= |
| | SUMMARY |
| | ================================================================= |
| | Consensus CV: 0.1771 |
| | Consensus eff_dim:109.5 |
| | Student v_cos: 0.686 |
| | Student v_cv: 0.182 |
| | Bank params: 2,921,088 |
| | Bank geo_eff_dim: 30.7 |
| | Bank geo_cv: 0.1892 |
| |
|
| | ================================================================= |
| | DONE |
| | ================================================================= |
| | ``` |
| | |
| | |
| | |
