| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - code-generation |
| - documentation |
| - text-to-code |
| - rust |
| - python |
| - javascript |
| - sparse-transformer |
| pipeline_tag: text-generation |
| inference: false |
| --- |
| |
| # Cofos Code Model (v1-logic) |
|
|
| **Cofos** is a logic- and coding-oriented decoder-only language model built on AMEFORGE's **SparseMind** architecture. |
| It is specialized for code generation, docstring construction, and code explanation in Python, JavaScript, and Rust. |
|
|
| Developed by **AMEFORGE**. |
|
|
| ## Architecture & Parameters |
| - **Architecture:** SparseMind v15 (dynamic neural type management, sparse attention, and sparse FFN) |
| - **Parameters:** 36,824,070 |
| - **Hidden dimension:** 384 |
| - **Layers:** 8 |
| - **Context length:** 512 tokens |
| - **Tokenizer:** Custom BPE Tokenizer (vocab size 16384) with atomic indents/operators/keywords |
|
|
| ## Training Details |
| - Trained on **30,000,000+** procedurally generated programming templates on-the-fly. |
| - Masked loss to learn code completion and documentation formatting. |
| - Best Python syntax-validity on held-out validation: 8.8% |
|
|
| ## How to use |
| ```python |
| import torch |
| import sentencepiece as spm |
| |
| # Load checkpoint |
| ckpt = torch.load("cofos_best.pt", map_location="cpu") |
| cfg_dict = ckpt["config"] |
| |
| # Instantiate model architecture |
| # model = SparseMind(Config(**cfg_dict)) |
| # model.load_state_dict(ckpt["model"]) |
| # model.eval() |
| ``` |
|
|
