tetu-AI/Tetu-Code-2

Têtu Code 2

A small coding agent trained from scratch, built to work in the terminal through OpenCode.

Têtu Code 2 is an independent research model with its own architecture. No weights from any other model went into it. A larger open model (Qwen3.6-35B-A3B, Apache 2.0) was used only as a teacher: it wrote demonstrations, and a program checked each one before training. The whole project ran on one consumer GPU (RTX 5070, 12 GB) plus about 30 hours of rented cloud GPU.

Model summary

Developed by	tetu-AI (independent project)
Model type	Decoder-only language model, custom architecture
Parameters	1.15B in total, of which 475M do the computation for each token. The rest is a large memory that the model reads at almost no cost.
Context length	65,536 tokens
Languages	Python first (plus HTML, CSS, JavaScript and others); instructions in English and French
Vocabulary	32,768 tokens, trained on code
Precision	bfloat16 (model.safetensors)
License	Apache 2.0

What it does:

• Works as an agent. It reads files, writes and edits them, and runs commands through OpenCode's tools.
• Thinks before acting. It writes a short reasoning between <|think|> and <|/think|> before each action.
• Knows when it may be wrong. A small head estimates, for every token, the probability that it is wrong.
• Comes with a server. It ships with an OpenAI-compatible server for OpenCode (1.x and 2.x tool formats). The server keeps the cache between turns and stops repeated identical calls.

Training

Stage	What
Pre-training	7.08B tokens: mostly code (Python, then web and other languages), plus code-related web pages, French and English web text and math
End of pre-training	The last quarter mixes in teacher projects, coding exercises whose solutions pass their tests, agent trajectories and commits
Agent fine-tuning	2,093 runs on real multi-file projects (command-line tools, games, sites, APIs, bug fixes, tests…). The teacher played them and a program checked them. Plus DAgger states: the model plays, and the teacher gives the right action.

Evaluation

All results come from measurements we had already made. Nothing was re-run for this page.

Code generation (EvalPlus)

Model	Size	Training tokens	HumanEval+	MBPP+
DeepSeek-Coder-1.3B-instruct	1.3B	2T	60.4	54.8
DeepSeek-Coder-1.3B-base	1.3B	2T	25.6	47.9
Qwen2.5-Coder-0.5B (base)	0.5B	5.5T	23.8	47.1
Têtu Code 1 (our previous model)	193M compute	7B	18.3	27.5
Têtu Code 2	475M compute	7.08B	15.9	30.7
SantaCoder-1.1B	1.1B	236B	14.0	–
StarCoderBase-1B	1B	1T	12.2	–
InCoder-1.3B	1.3B	~60B	11.0	–
CodeGen2-1B	1B	~400B	9.1	–

• Têtu Code: EvalPlus 0.3.1, greedy decoding, one sample per problem, chat prompt. A second run with the same setup gave HumanEval+ 14.6 and MBPP+ 30.7, so allow for about ±1.5 points of noise.
• Other models: official EvalPlus leaderboard, read on 23 Oct. 2026; Qwen2.5-Coder numbers come from its authors.
• Reading the table: Têtu Code 2 beats the 1B code models of 2023 (SantaCoder, StarCoderBase-1B) after seeing 30 to 140 times fewer tokens. It is behind DeepSeek-Coder-1.3B and Qwen2.5-Coder-0.5B, which saw 280 to 780 times more tokens.

As an agent

	Têtu Code 1	Têtu Code 2	Teacher (Qwen3.6-35B-A3B)
40 held-out real projects in OpenCode 2.1.4 (hidden tests)	0/40	8/40	29/40
Older single-function agent benchmark (OpenCode 1.19.4)	46.5 %	28.5 %	85.5 %
Validation loss on Python (pre-training)	0.821	0.777	–

Speed on an RTX 5070: about 143 tokens/s at 2k context and 71 tokens/s at 16k.

Limitations

• It is a small model. It often gets the logic wrong from the start, or invents variables.
• It does not repair its own failing tests. When its tests fail, it tends to re-run commands instead of fixing the code.
• It follows multi-turn conversations poorly. It can miss a correction or a second request given later in the same conversation.
• It is an agent, not a chatbot. In the plain terminal chat (chat.py), it often still tries to call a tool. Ask for it through OpenCode.
• It sometimes writes files in the wrong folder. The bundled server moves such paths back into the project.
• Do not let it run unattended on code that matters. Use a sandbox or a throwaway directory.

Usage

The architecture is custom, so it does not load with transformers. Use the bundled inference code.

git clone https://huggingface.co/tetu-AI/Tetu-Code-2 && cd Tetu-Code-2
pip install -r requirements.txt
python chat.py                                   # terminal chat, no tools (--cpu without a GPU)
python -m tetu_code.serve --model . --port 8070  # OpenAI-compatible server for OpenCode

opencode.json:

"provider": {"tetu": {"npm": "@ai-sdk/openai-compatible", "name": "Têtu Code",
  "options": {"baseURL": "http://127.0.0.1:8070/v1", "apiKey": "x"},
  "models": {"tetu-code": {"name": "Têtu Code", "tool_call": true,
                           "limit": {"context": 65536, "output": 4096}}}}}

From Python:

import torch
from tetu_code.generate import Decoder, load
model, tok = load(".", device="cuda")          # or device="cpu", dtype=torch.float32
dec = Decoder(model, tok, max_len=16384)

License

Apache 2.0.

Teachers, both Apache 2.0:

• Qwen3.6-35B-A3B, for the agent runs;
• Qwen2.5-7B-Instruct, for everyday conversations.

Training data:

• github-code-clean (permissively licensed files only);
• FineWeb-Edu, FineWeb2-HQ and FineMath (ODC-By);
• OpenCodeInstruct;
• Nemotron-SWE trajectories;
• CommitPackFT.

Check each dataset's own terms before commercial use.

Citation

@misc{tetucode2026,
  title  = {Têtu Code 2: a small coding agent trained from scratch},
  author = {tetu-AI},
  year   = {2026},
  url    = {https://huggingface.co/tetu-AI/Tetu-Code-2}
}