---
language: en
library_name: transformers
base_model: Qwen/Qwen2.5-3B-Instruct
pipeline_tag: text-generation
tags:
- qwen
- qwen2.5
- 3b
- lora
- coding
- code
- software-engineering
license: apache-2.0
---

# Ult1-Coding

A 3-billion-parameter coding specialist -- master-level software engineer.

Based on Qwen2.5-3B-Instruct with an embedded master programmer system prompt containing few-shot coding demonstrations and a coding-focused LoRA adapter (rank 16, 8 target module types).

Usage:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1-coding")
tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1-coding")

messages = [{"role": "user", "content": "Write a Python async web scraper with retry logic"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

The system prompt with few-shot examples is auto-injected by the chat template - no manual system prompt needed.

GGUF: Download Ult1-Coding-Q8_0.gguf for CPU inference with llama.cpp.

Training Data: training_data.json contains 10 coding Q&A pairs (Python, JavaScript, Rust, SQL, TypeScript, Go). Use with train.py on a GPU.

Details:
- Base: Qwen2.5-3B-Instruct (3B params)
- LoRA: Rank 16, targets q/k/v/o + gate/up/down projections
- Context: 32,768 tokens
- Focus: Code generation, algorithms, system design, debugging