---
license: apache-2.0
language:
- en
tags:
- code-generation
- documentation
- text-to-code
- rust
- python
- javascript
- sparse-transformer
pipeline_tag: text-generation
inference: false
---

# Cofos Code Model (v1-logic)

**Cofos** is a logic- and coding-oriented decoder-only language model built on AMEFORGE's **SparseMind** architecture.
It is specialized for code generation, docstring construction, and code explanation in Python, JavaScript, and Rust.

Developed by **AMEFORGE**.

## Architecture & Parameters
- **Architecture:** SparseMind v15 (dynamic neural type management, sparse attention, and sparse FFN)
- **Parameters:** 36,824,070
- **Hidden dimension:** 384
- **Layers:** 8
- **Context length:** 512 tokens
- **Tokenizer:** Custom BPE Tokenizer (vocab size 16384) with atomic indents/operators/keywords

## Training Details
- Trained on **30,000,000+** procedurally generated programming templates on-the-fly.
- Masked loss to learn code completion and documentation formatting.
- Best Python syntax-validity on held-out validation: 8.8%

## How to use
```python
import torch
import sentencepiece as spm

# Load checkpoint
ckpt = torch.load("cofos_best.pt", map_location="cpu")
cfg_dict = ckpt["config"]

# Instantiate model architecture
# model = SparseMind(Config(**cfg_dict))
# model.load_state_dict(ckpt["model"])
# model.eval()
```