---
license: apache-2.0
base_model: unsloth/Qwen2.5-Coder-7B-Instruct
tags:
- code
- leetcode
- cpp
- code-generation
- competitive-programming
- qwen2.5-coder
- dora
- qdora
- weight-decomposed-lora
- instruction-tuned
- sft
- algorithm-generation
- function-generation
- coding-assistant
- on-device
- gguf
- ollama
- vllm
- text-generation-inference
- doocs-leetcode
- synthetic-verification
- quantized
- algorithms
language:
- en
library_name: peft
pipeline_tag: text-generation
datasets:
- AmareshHebbar/leetcode-code-gen-datasets
co2_eq_emissions:
emissions: 0
source: "estimate, not measured with a carbon-tracking tool"
training_type: "fine-tuning"
geographical_location: "EU-West"
hardware_used: "NVIDIA A40 (48GB)"
model-index:
- name: leetcode-cpp-qwen25-coder-7b
results: []
---
# ⚙️ LeetCode C++ Coder
### Qwen2.5-Coder-7B, QDoRA fine-tuned to solve LeetCode problems in C++
[](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b)
[](https://huggingface.co/datasets/AmareshHebbar/leetcode-code-gen-datasets)
[](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF)
[](https://www.apache.org/licenses/LICENSE-2.0)
[](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct)
[](#why-qdora)
[](#ollama)
[](#vllm)
[](#tgi)
*Part of the [LeetCode Multi-Language Coder Suite](https://huggingface.co/collections/AmareshHebbar/leetcode-multi-language-coder-suite) — 4 language specialists, one base model, one pipeline*
---
## TL;DR
Given a LeetCode-style problem statement, its sample input/output, and an algorithm tag, generates a working C++ solution.
```
PROBLEM: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
ALGORITHM: Hash Map
OUTPUT (C++):
class Solution {
public:
vector twoSum(vector& nums, int target) {
unordered_map seen;
for (int i = 0; i < nums.size(); i++) {
if (seen.count(target - nums[i])) return {seen[target - nums[i]], i};
seen[nums[i]] = i;
}
return {};
}
};
```
| | |
|---|---|
| **Base model** | [unsloth/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct) |
| **Method** | QDoRA (quantized DoRA, not plain LoRA) |
| **Training data** | [leetcode-code-gen-datasets](https://huggingface.co/datasets/AmareshHebbar/leetcode-code-gen-datasets) config `cpp` |
| **Data provenance** | scraped from [doocs/leetcode](https://github.com/doocs/leetcode) (3,977 problems), execution-verified, no synthetic/LLM-generated solutions |
| **Data quality** | execution-checked against sample I/O (see dataset card for exact rate) |
| **Weights here** | QDoRA adapter only (~160MB) — load on top of the base model |
| **GGUF build** | [leetcode-cpp-qwen25-coder-7b-GGUF](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF) — q4_k_m / q5_k_m / q8_0 |
| **License** | Apache 2.0 |
---
## Why QDoRA {#why-qdora}
DoRA splits each adapted weight into magnitude + direction and trains both, which follows full fine-tuning's behavior more closely than plain LoRA — important for code where small precision errors break correctness outright. 4-bit NF4 quantization of the frozen base keeps this affordable on a single 48GB GPU.
Concretely, versus the plain-QLoRA v1 release of this suite: DoRA adds a per-column
trainable magnitude vector on top of the usual low-rank direction update, so the
adapter can rescale a feature's importance instead of only rotating it. On a code
task where a single wrong operator or dropped edge case fails the whole solution,
that closer match to full fine-tuning's update pattern showed up as fewer
near-miss failures during our own qualitative review, at the same LoRA rank and
VRAM budget.
```python
# training-side PEFT config (see build_language_datasets.py / trainer script for full pipeline)
from peft import LoraConfig
peft_config = LoraConfig(
r=16,
lora_alpha=32,
lora_dropout=0.0,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
use_dora=True, # <- this is what makes it QDoRA, not QLoRA
task_type="CAUSAL_LM",
)
```
---
## Benchmarks (free, reproducible)
Run `benchmark_suite.py` from the deployment kit to reproduce. All numbers are pass@1 unless noted.
| Benchmark | Language | Pass@1 | Pass@10 | Notes |
|---|---|---|---|---|
| [HumanEval-X](https://huggingface.co/datasets/THUDM/humaneval-x) | C++ | _run benchmark_suite.py_ | _run benchmark_suite.py_ | 164 problems, execution-verified |
| [MultiPL-E](https://huggingface.co/datasets/nuprl/MultiPL-E) (HumanEval subset) | C++ | _run benchmark_suite.py_ | — | cross-check vs HumanEval-X |
| Held-out LeetCode test split | C++ | _run benchmark_suite.py_ | — | from `leetcode-code-gen-datasets` (`cpp`) test split, exact I/O match |
| Tokens/sec (fp16, A40) | C++ | — | — | latency benchmark, see script |
| Tokens/sec (GGUF q4_k_m, CPU) | C++ | — | — | latency benchmark, see script |
> Numbers are intentionally left blank in this template — `benchmark_suite.py` fills a `results/leetcode-cpp-qwen25-coder-7b.json` file and this table should be regenerated from it.
---
## Intended use
Drop-in solution generator for C++ coding-practice tools, interview-prep apps, and automated code-review sandboxes for algorithmic problems.
### Direct use
Give a problem statement (+ optional algorithm hint), get back a C++ function/class implementing it.
### Downstream use
Feed output into an automated grader (run against test cases), a code-review bot, or a practice-app "show solution" feature.
### Out of scope
- Production system design or non-algorithmic code (this model specializes narrowly on LeetCode-style problems)
- Security-critical code without human review
- Guaranteed-optimal complexity — treat output as a strong first draft, not a proof
---
## Quickstart
### Option A — Transformers + PEFT
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base_model = "unsloth/Qwen2.5-Coder-7B-Instruct"
adapter = "AmareshHebbar/leetcode-cpp-qwen25-coder-7b"
tokenizer = AutoTokenizer.from_pretrained("AmareshHebbar/leetcode-cpp-qwen25-coder-7b")
model = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)
messages = [
{"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
{"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
```
### Batch inference (many problems at once)
```python
problems = [
"Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
"Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window",
"Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head",
]
prompts = [
tokenizer.apply_chat_template(
[{"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."}, {"role": "user", "content": p}],
tokenize=False, add_generation_prompt=True,
)
for p in problems
]
tokenizer.padding_side = "left"
batch = tokenizer(prompts, return_tensors="pt", padding=True).to(model.device)
outputs = model.generate(**batch, max_new_tokens=512, temperature=0.2, do_sample=True)
for i, o in enumerate(outputs):
print(f"--- solution {i} ---")
print(tokenizer.decode(o[batch['input_ids'].shape[1]:], skip_special_tokens=True))
```
### Streaming output (token-by-token)
```python
from transformers import TextIteratorStreamer
from threading import Thread
streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
gen_kwargs = dict(input_ids=inputs, max_new_tokens=512, temperature=0.2, do_sample=True, streamer=streamer)
Thread(target=model.generate, kwargs=gen_kwargs).start()
for token in streamer:
print(token, end="", flush=True)
```
### Structured JSON output (code + complexity + explanation)
```python
json_system_prompt = (
"You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution. "
'Respond ONLY with JSON: {"code": "...", "time_complexity": "...", '
'"space_complexity": "...", "explanation": "..."}'
)
messages = [
{"role": "system", "content": json_system_prompt},
{"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
raw = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
import json
result = json.loads(raw.strip().removeprefix("```json").removesuffix("```").strip())
print(result["code"])
print(result["time_complexity"], result["space_complexity"])
```
### Option B — Unsloth (2x faster load + inference)
```python
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="AmareshHebbar/leetcode-cpp-qwen25-coder-7b",
max_seq_length=2048,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
messages = [
{"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
{"role": "user", "content": "Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
### Option C — vLLM (production serving, OpenAI-compatible) {#vllm}
```bash
vllm serve unsloth/Qwen2.5-Coder-7B-Instruct \
--enable-lora \
--lora-modules leetcode-cpp-qwen25-coder-7b=AmareshHebbar/leetcode-cpp-qwen25-coder-7b \
--host 0.0.0.0 --port 8000 --dtype bfloat16
```
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
response = client.chat.completions.create(
model="leetcode-cpp-qwen25-coder-7b",
messages=[
{"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
{"role": "user", "content": "Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head"},
],
temperature=0.2,
)
print(response.choices[0].message.content)
```
Streaming with vLLM's OpenAI-compatible endpoint:
```python
stream = client.chat.completions.create(
model="leetcode-cpp-qwen25-coder-7b",
messages=[{"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```
### Option D — TGI (Text Generation Inference) {#tgi}
```bash
docker run --gpus all --shm-size 1g -p 8080:80 \
-v $PWD/data:/data ghcr.io/huggingface/text-generation-inference:latest \
--model-id unsloth/Qwen2.5-Coder-7B-Instruct \
--lora-adapters leetcode-cpp-qwen25-coder-7b=AmareshHebbar/leetcode-cpp-qwen25-coder-7b
```
```bash
curl 127.0.0.1:8080/generate_stream \
-X POST \
-d '{"inputs":"<|im_start|>system\nYou are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map<|im_end|>\n<|im_start|>assistant\n","parameters":{"max_new_tokens":512}}' \
-H 'Content-Type: application/json'
```
### Option E — Ollama (local, mobile/edge-friendly) {#ollama}
```bash
# 1. Pull the GGUF build
huggingface-cli download AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF leetcode-cpp-qwen25-coder-7b.q4_k_m.gguf --local-dir .
# 2. Create the model from the Modelfile shipped in the deployment kit (see deploy_ollama.py)
ollama create leetcode-cpp-qwen25-coder-7b -f Modelfile.cpp
# 3. Run it
ollama run leetcode-cpp-qwen25-coder-7b "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"
```
Python client against a local Ollama server:
```python
import requests
r = requests.post("http://localhost:11434/api/generate", json={
"model": "leetcode-cpp-qwen25-coder-7b",
"prompt": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
"stream": False,
})
print(r.json()["response"])
```
### Option F — GGUF / llama.cpp direct (mobile/edge inference)
```bash
./llama-cli -m leetcode-cpp-qwen25-coder-7b.q4_k_m.gguf \
-p "<|im_start|>system\nYou are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.<|im_end|>\n<|im_start|>assistant\n" \
-n 512 --temp 0.2
```
See `export_gguf.py` in the deployment kit for building q4_k_m / q5_k_m / q8_0 variants, and the mobile integration notes there for Android (llama.cpp JNI) and iOS (llama.cpp via Swift bindings).
---
## Training details
### Why this base model
Qwen2.5-Coder-7B-Instruct was chosen over a general instruct model because its
pretraining already concentrates capacity on code — the QDoRA adapter only has to
specialize output format and LeetCode-specific conventions (function signatures,
in-place vs. new-array conventions, C++ idioms) rather than teach the model
to code from scratch. 7B was picked as the size that still fits comfortably in a
single-GPU QDoRA run while keeping enough headroom that the base model's code
reasoning survives adaptation.
### Data pipeline
Source: [doocs/leetcode](https://github.com/doocs/leetcode), 3,977 problems with
English documentation. Each problem can have multiple solutions spanning different
algorithm tags (greedy, DP, two pointers, etc.) — the pipeline treats this as a
one-to-many problem-to-solution structure rather than picking a single "canonical" answer.
| Stage | What it does |
|---|---|
| `extract_doocs.py` | pulls problem statement + I/O examples + per-solution algorithm tag from doocs/leetcode |
| `verify.py` | executes each extracted solution against its sample I/O, drops anything that fails |
| `normalize.py` | standardizes formatting/whitespace and problem/solution schema across all 4 languages |
| `build_language_datasets.py` | splits into per-language configs and writes the final train/val/test SFT rows |
execution-checked against sample I/O (see dataset card for exact rate). Full extraction/verification/build code lives alongside the
[leetcode-code-gen-datasets](https://huggingface.co/datasets/AmareshHebbar/leetcode-code-gen-datasets) dataset card.
### Hyperparameters
| Parameter | Value |
|---|---|
| Method | QDoRA (`use_dora=True` in PEFT's `LoraConfig`) |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Base quantization | 4-bit NF4 |
| Max sequence length | 2048 |
| Optimizer | paged_adamw_8bit |
| LR schedule | 2e-4, cosine |
### Training compute
| | |
|---|---|
| **GPU** | NVIDIA A40 (48GB) |
| **Cloud provider** | RunPod |
| **CO2 estimate** | self-reported, not measured with a carbon tracker — treat as approximate |
Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) + TRL's `SFTTrainer`,
DoRA enabled via PEFT.
---
## Bias, risks & limitations
**Narrow specialization.** This model is tuned tightly on LeetCode-style algorithmic problems — general software-engineering code (frameworks, infra, business logic) is out of distribution.
**Verify before trusting.** Like any LLM, generated solutions can look plausible and still fail an edge case (empty input, integer overflow, off-by-one). Always run against test cases before use.
**Not exhaustive on complexity.** The model doesn't guarantee asymptotically optimal solutions — check the complexity claims yourself for performance-sensitive use.
**Data recency.** Reflects the state of `doocs/leetcode` at the time of extraction — newer problems added to LeetCode after that snapshot won't be covered.
---
## FAQ
**Q: Can I merge the adapter into the base model?**
Yes — `model.merge_and_unload()` after loading with PEFT, or Unsloth's `save_pretrained_merged()`. DoRA adapters merge the same way LoRA adapters do.
**Q: Why QDoRA instead of plain QLoRA?**
See [Why QDoRA](#why-qdora) above — short version: DoRA's magnitude/direction split tracks full fine-tuning more closely, which matters for code correctness.
**Q: Why QDoRA instead of full fine-tuning?**
Qwen2.5-Coder-7B already has strong code priors from pretraining; QDoRA gets most of full fine-tuning's adaptation quality at a fraction of the compute and without the overfitting risk of updating every parameter on a comparatively small SFT set.
**Q: Which quantization should I use on mobile?**
q4_k_m is the best size/quality tradeoff for phones; q5_k_m if you have RAM headroom; avoid q2/q3 for code generation — correctness drops sharply below 4-bit.
**Q: Does this model store or transmit my input?**
No — inference runs entirely on whatever infrastructure you deploy it to.
---
## Related models in this suite
| Model | Language |
|---|---|
| [leetcode-python-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-python-qwen25-coder-7b) | Python |
| [leetcode-java-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-java-qwen25-coder-7b) | Java |
| [leetcode-cpp-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b) | C++ (this model) |
| [leetcode-javascript-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-javascript-qwen25-coder-7b) | JavaScript |
**Full collection:** [LeetCode Multi-Language Coder Suite](https://huggingface.co/collections/AmareshHebbar/leetcode-multi-language-coder-suite)
---
## Changelog
| Version | Notes |
|---|---|
| v3.0 | Switched to QDoRA, added rationale + PEFT config, batch/streaming/JSON inference samples, expanded tags |
| v2.0 | Added GGUF builds, Ollama/vLLM/TGI deployment, benchmark harness (HumanEval-X, MultiPL-E, held-out test split) |
| v1.0 | Initial release — QLoRA fine-tune |
---
## Citation
```bibtex
@misc{leetcodecoder2026,
author = {Hebbar, Amaresh},
title = {LeetCode Multi-Language Coder Suite},
year = {2026},
publisher = {HuggingFace},
url = {https://huggingface.co/AmareshHebbar}
}
```
## Contact
[](https://github.com/amareshhebbar)
[](https://www.linkedin.com/in/gvamaresh)
[](https://huggingface.co/AmareshHebbar)