Modelo Light preentrenado (viejita ~1M params) + tokenizer + card

README.md

@@ -0,0 +1,40 @@
+---
+language: es, en
+license: mit
+tags:
+  - light
+  - llm
+  - from-scratch
+  - spanish
+---
+
+# ⚡ Light — mini-LLM entrenada desde cero
+
+Modelo entrenado con el proyecto [light](https://github.com/jmarc9901/light).
+
+- Arquitectura: transformer decoder-only (RoPE + RMSNorm + SwiGLU), desde cero
+- Parámetros: 1,196,672
+- Tamaños: 4 capas, 128 dim, 1024 tokens, contexto 192
+- Tokenizer: byte-level BPE con pre-tokenización por palabras
+
+Para generar texto:
+
+```python
+import torch
+from tokenizer import BPETokenizer
+from model import GPT
+from config import ModelConfig
+
+ckpt = torch.load("best.ckpt", map_location="cpu", weights_only=True)
+model = GPT(ModelConfig(**ckpt["model_cfg"])).eval()
+model.load_state_dict(ckpt["model_state"])
+tok = BPETokenizer.load("tokenizer.json")
+
+ids = tok.encode("En un lugar de la Mancha")
+idx = torch.tensor([ids])
+out = model.generate(idx, 200, temperature=0.7, top_p=0.95, repetition_penalty=1.1)
+print(tok.decode(out[0].tolist()))
+```
+
+## Limitaciones
+Modelo pequeño de aprendizaje: imita el estilo de su corpus; no conversa ni responde preguntas.

best.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cb4c4948693583f29d42d2b07902f080de3c380119aad016d8b7b81ae03c88e
+size 11640449

tokenizer.json

@@ -0,0 +1 @@
+{"vocab_size": 1024, "pretokenize": true, "merges": [[117, 101], [32, 100], [32, 101], [32, 108], [32, 97], [111, 115], [113, 256], [257, 101], [32, 115], [101, 114], [32, 99], [32, 112], [101, 110], [97, 115], [97, 110], [32, 109], [32, 262], [111, 110], [97, 114], [101, 115], [111, 114], [32, 116], [32, 121], [100, 111], [195, 173], [32, 104], [258, 110], [115, 116], [97, 108], [32, 110], [32, 118], [259, 97], [114, 97], [114, 101], [105, 110], [258, 108], [97, 98], [99, 105], [117, 110], [195, 179], [97, 100], [42, 42], [116, 101], [264, 101], [111, 109], [266, 273], [195, 169], [32, 102], [195, 161], [117, 105], [108, 97], [99, 104], [264, 117], [285, 111], [116, 111], [32, 98], [100, 105], [267, 276], [32, 111], [258, 283], [105, 268], [280, 97], [258, 115], [100, 101], [108, 101], [114, 111], [114, 105], [257, 105], [260, 108], [259, 261], [116, 97], [226, 128], [327, 148], [106, 111], [259, 111], [195, 177], [32, 294], [32, 103], [100, 97], [116, 105], [259, 101], [271, 105], [115, 101], [45, 45], [271, 101], [266, 300], [108, 111], [259, 269], [97, 109], [265, 111], [277, 111], [99, 101], [267, 274], [307, 111], [32, 32], [103, 111], [113, 305], [114, 275], [281, 97], [266, 117], [263, 108], [264, 105], [271, 117], [111, 108], [42, 42], [297, 360], [32, 361], [32, 83], [270, 279], [97, 279], [257, 273], [32, 289], [295, 110], [304, 115], [286, 105], [263, 115], [101, 108], [341, 111], [258, 116], [268, 116], [118, 105], [281, 292], [109, 112], [115, 105], [32, 32], [99, 97], [114, 270], [32, 352], [100, 296], [256, 110], [256, 283], [99, 111], [122, 97], [99, 117], [256, 114], [292, 97], [265, 97], [100, 261], [116, 114], [109, 111], [109, 101], [32, 32], [32, 68], [296, 97], [32, 80], [277, 270], [278, 111], [99, 265], [105, 114], [113, 117], [32, 69], [271, 369], [46, 46], [284, 108], [264, 290], [111, 98], [276, 97], [32, 194], [32, 32], [108, 105], [81, 305], [103, 294], [260, 110], [348, 97], [109, 97], [116, 275], [302, 108], [264, 265], [117, 115], [109, 105], [268, 298], [32, 35], [108, 261], [262, 108], [103, 117], [363, 270], [100, 275], [109, 261], [271, 270], [32, 32], [286, 265], [105, 101], [282, 116], [267, 111], [416, 329], [440, 298], [105, 265], [332, 97], [114, 269], [277, 268], [299, 331], [32, 290], [267, 256], [266, 292], [32, 65], [277, 101], [293, 97], [116, 274], [32, 67], [116, 117], [32, 328], [32, 441], [431, 349], [260, 429], [105, 111], [32, 32], [285, 105], [116, 288], [32, 422], [303, 256], [108, 269], [260, 115], [304, 110], [266, 261], [32, 105], [195, 186], [311, 385], [313, 262], [105, 97], [100, 269], [308, 115], [103, 97], [116, 261], [277, 117], [100, 265], [99, 275], [311, 316], [32, 77], [32, 32], [281, 105], [116, 321], [274, 97], [112, 273], [32, 76], [99, 273], [266, 97], [266, 269], [98, 97], [105, 115], [271, 284], [270, 298], [315, 111], [333, 382], [346, 279], [302, 110], [32, 106], [281, 101], [267, 265], [105, 261], [114, 261], [286, 101], [103, 274], [275, 101], [413, 191], [116, 268], [110, 111], [312, 295], [116, 269], [112, 97], [293, 368], [280, 270], [122, 111], [286, 268], [32, 32], [267, 269], [446, 276], [358, 121], [377, 317], [353, 488], [264, 111], [118, 101], [259, 320], [260, 99], [259, 105], [32, 405], [98, 105], [277, 288], [260, 114], [112, 101], [98, 289], [261, 111], [46, 339], [32, 84], [116, 265], [258, 288], [316, 310], [115, 111], [264, 292], [32, 40], [10, 10], [290, 111], [273, 275], [115, 97], [293, 111], [32, 89], [281, 300], [271, 97], [32, 114], [386, 288], [316, 279], [118, 111], [32, 32], [312, 329], [337, 115], [285, 273], [32, 78], [112, 265], [267, 321], [323, 329], [117, 302], [303, 97], [118, 265], [271, 265], [264, 284], [307, 97], [109, 270], [99, 274], [100, 274], [315, 97], [296, 261], [467, 280], [290, 97], [10, 350], [99, 270], [315, 101], [347, 100], [283, 111], [117, 288], [32, 32], [112, 274], [283, 101], [355, 284], [102, 105], [106, 261], [32, 34], [32, 288], [449, 409], [408, 46], [346, 393], [265, 273], [106, 101], [271, 268], [348, 101], [524, 512], [267, 101], [303, 105], [117, 108], [106, 97], [355, 364], [32, 32], [267, 268], [293, 114], [112, 111], [99, 268], [411, 289], [267, 345], [328, 44], [278, 97], [108, 306], [108, 274], [276, 275], [32, 82], [271, 269], [271, 280], [277, 284], [99, 269], [292, 270], [324, 417], [100, 268], [448, 115], [291, 306], [267, 97], [99, 317], [116, 296], [32, 32], [104, 101], [383, 268], [266, 276], [267, 117], [268, 279], [398, 504], [257, 117], [99, 300], [108, 270], [316, 101], [271, 111], [394, 261], [264, 273], [109, 426], [265, 101], [257, 261], [267, 114], [264, 97], [469, 97], [100, 114], [568, 581], [322, 109], [108, 275], [109, 276], [413, 161], [471, 110], [263, 607], [105, 279], [257, 274], [267, 289], [296, 289], [260, 294], [326, 279], [32, 32], [112, 276], [265, 261], [266, 274], [266, 344], [98, 101], [269, 101], [109, 268], [110, 101], [256, 108], [256, 115], [286, 359], [263, 106], [410, 111], [286, 554], [114, 274], [281, 269], [331, 111], [116, 295], [530, 302], [281, 265], [32, 95], [314, 486], [260, 112], [333, 117], [285, 290], [115, 117], [32, 32], [314, 463], [283, 97], [354, 403], [109, 541], [358, 349], [121, 97], [112, 105], [286, 261], [270, 388], [438, 289], [109, 98], [105, 103], [406, 116], [99, 290], [267, 273], [366, 319], [277, 300], [436, 384], [294, 279], [437, 378], [103, 288], [259, 306], [318, 112], [469, 269], [492, 97], [116, 276], [270, 310], [270, 97], [281, 117], [418, 100], [260, 100], [99, 261], [331, 97], [32, 276], [32, 32], [318, 389], [101, 378], [405, 280], [259, 117], [109, 269], [32, 70], [106, 265], [358, 307], [274, 273], [280, 110], [32, 86], [438, 268], [105, 108], [296, 269], [260, 98], [114, 117], [264, 268], [394, 269], [32, 321], [307, 101], [285, 261], [118, 97], [323, 115], [45, 45], [121, 111], [103, 656], [339, 750], [314, 102], [591, 122], [284, 275], [32, 261], [264, 610], [108, 276], [267, 284], [529, 98], [99, 322], [344, 261], [285, 101], [303, 390], [303, 290], [118, 268], [277, 711], [430, 97], [32, 32], [256, 351], [271, 710], [110, 97], [116, 270], [260, 425], [99, 289], [439, 387], [104, 97], [117, 114], [311, 97], [324, 108], [106, 276], [401, 310], [370, 334], [354, 121], [266, 289], [103, 305], [271, 275], [266, 322], [315, 304], [32, 71], [277, 631], [315, 391], [268, 452], [280, 269], [283, 105], [270, 111], [314, 98], [319, 108], [561, 111], [472, 97], [267, 353], [768, 111], [271, 390], [32, 66], [346, 475], [101, 122], [118, 375], [277, 640], [344, 426], [267, 105], [111, 122], [265, 110], [32, 322], [296, 276], [386, 321], [69, 76], [303, 284], [32, 32], [606, 115], [346, 334], [662, 262], [501, 117], [293, 273], [100, 276], [459, 306], [293, 295], [114, 317], [105, 384], [260, 729], [114, 273], [257, 111], [371, 112], [275, 335], [592, 345], [445, 351], [103, 261], [100, 273], [258, 120], [270, 421], [393, 101], [267, 652], [527, 118], [286, 284], [99, 276], [340, 782], [266, 111], [100, 317], [99, 292], [102, 101], [99, 295], [32, 753], [257, 97], [348, 298], [470, 378], [434, 392], [277, 442], [103, 105], [303, 101], [259, 275], [286, 97], [98, 320], [115, 273], [260, 395], [341, 112], [332, 111], [271, 276], [448, 319], [101, 388], [281, 270], [565, 115], [314, 642], [32, 47], [32, 73], [101, 306], [264, 359], [333, 288], [32, 353], [109, 117], [122, 269], [285, 97], [377, 265], [358, 733], [32, 32], [263, 338], [680, 326], [323, 349], [257, 317], [446, 412], [386, 111], [109, 274], [260, 104], [527, 103], [260, 121], [303, 276], [394, 274], [334, 279], [264, 302], [689, 417], [266, 390], [593, 46], [730, 507], [314, 589], [375, 275], [259, 771], [312, 111], [69, 116], [100, 117], [367, 121], [781, 280], [300, 101], [623, 97], [266, 270], [266, 385], [291, 428], [320, 103], [267, 437], [857, 288], [293, 635], [117, 97], [98, 111], [122, 274], [116, 284], [275, 116], [445, 317], [281, 409], [267, 661], [314, 121], [335, 108], [286, 375], [597, 261], [294, 310], [551, 672], [434, 111], [32, 119], [311, 274], [112, 415], [194, 187], [32, 72], [301, 338], [266, 437], [301, 319], [105, 334], [32, 49], [293, 261], [108, 268], [294, 381], [302, 494], [465, 288], [331, 261], [277, 276], [100, 295], [41, 46], [293, 279], [32, 32], [114, 268], [112, 268], [299, 97], [293, 269], [312, 279], [406, 110], [311, 111], [118, 261], [468, 279], [760, 292], [292, 320], [103, 268], [118, 270], [525, 121], [281, 273], [637, 262], [551, 535], [266, 442], [314, 744], [115, 268], [266, 284], [273, 460], [294, 97], [301, 111], [288, 279], [500, 279], [264, 280], [104, 105], [424, 111], [106, 269], [108, 117], [285, 256], [293, 547], [261, 105], [106, 274], [267, 306], [271, 274], [331, 276], [295, 395], [101, 97], [283, 276], [265, 275], [506, 122], [103, 270], [318, 562], [10, 32], [32, 74], [32, 404], [660, 417], [101, 116], [543, 265], [684, 433], [552, 121], [688, 274], [277, 105], [315, 269], [450, 108], [73, 73], [122, 261], [109, 369], [69, 77], [318, 111], [333, 411], [398, 101], [266, 105], [277, 344], [728, 289], [122, 368]]}