shadow head on the FULL FP body v2 (lloyd 3, cosine, batch 4->8, polyak 200, init shadow_head.safetensors): held-out KL 0.0788->0.0470, wikitext probe 1.0740x, 1500 steps
Browse files
shadow_head_fp_v2.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d922211f3dade3f501cbd6b78970ecc610bc9b66e2a64ccda77a7ce9d3abc865
|
| 3 |
+
size 5085593776
|