Aurora-Code-Mini-1

Compact. Capable. Completely open.

Aurora-Code-Mini-1 is a 12B dense coding model built by Perciqa, a Canadian AI company. Fine-tuned from Gemma 4 12B on curated agentic coding instruction pairs using QLoRA.

Training Details

Field	Value
Base Model	google/gemma-4-12b
Method	QLoRA (4-bit NF4)
LoRA Rank	16
LoRA Alpha	32
LoRA Dropout	0.0
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Precision	BF16 compute, NF4 storage
Context Length	2048 tokens
Epochs	3
Batch Size	2 (effective 16 with grad accum 8)
Learning Rate	2e-4
Warmup Steps	10
Optimizer	AdamW
Gradient Checkpointing	True

Training Data

• Source: perciqa/aurora-code-sft-v1
• Train: 1,992 records
• Validation: 225 records
• Categories: Code generation, debugging, code review, agentic coding, refactoring, test writing

Hardware

• GPU: AMD Radeon W7900 (48 GB VRAM)
• Framework: ROCm 7.2.4, PyTorch 2.10.0, Transformers 5.14.1, TRL 0.24.0, PEFT 0.19.1
• Training time: ~10 hours

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Perciqa/Aurora-Code-Mini-1"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto",
)

system_prompt = (
    "You are Aurora, an AI code assistant built by Perciqa. "
    "You help developers write, review, and understand code. "
    "You provide clear, correct, and complete solutions. "
    "When you're unsure, you say so."
)

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Write a Python function to merge two sorted lists."},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

License

Apache 2.0

About Perciqa

Perciqa is a Canadian AI company building enterprise models and tools that organisations can deploy, audit, and fully control — on their own infrastructure, on their own terms.

perciqa.com