README.md

---
license: apache-2.0
base_model: unsloth/Qwen2.5-Coder-7B-Instruct
tags:
  - code
  - leetcode
  - cpp
  - code-generation
  - competitive-programming
  - qwen2.5-coder
  - dora
  - qdora
  - weight-decomposed-lora
  - instruction-tuned
  - sft
  - algorithm-generation
  - function-generation
  - coding-assistant
  - on-device
  - gguf
  - ollama
  - vllm
  - text-generation-inference
  - doocs-leetcode
  - synthetic-verification
  - quantized
  - algorithms
language:
  - en
library_name: peft
pipeline_tag: text-generation
datasets:
  - AmareshHebbar/leetcode-codegen-cpp
co2_eq_emissions:
  emissions: 0
  source: "estimate, not measured with a carbon-tracking tool"
  training_type: "fine-tuning"
  geographical_location: "EU-West"
  hardware_used: "NVIDIA A40 (48GB)"
model-index:
  - name: leetcode-cpp-qwen25-coder-7b
    results: []
---

<div align="center">

# ⚙️ LeetCode C++ Coder
### Qwen2.5-Coder-7B, QDoRA fine-tuned to solve LeetCode problems in C++

[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Model-leetcode--cpp--qwen25--coder--7b-FFD21E)](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b)
[![Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-leetcode--codegen--cpp-blue)](https://huggingface.co/datasets/AmareshHebbar/leetcode-codegen-cpp)
[![GGUF](https://img.shields.io/badge/GGUF-quantized-6f42c1)](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF)
[![License](https://img.shields.io/badge/license-Apache%202.0-green)](https://www.apache.org/licenses/LICENSE-2.0)
[![Base Model](https://img.shields.io/badge/base-Qwen2.5--Coder--7B-orange)](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct)
[![Method](https://img.shields.io/badge/method-QDoRA-critical)](#why-qdora)
[![Ollama](https://img.shields.io/badge/-Ollama-000000?logo=ollama)](#ollama)
[![vLLM](https://img.shields.io/badge/-vLLM-333333)](#vllm)
[![TGI](https://img.shields.io/badge/-TGI-yellow)](#tgi)

*Part of the [LeetCode Multi-Language Coder Suite](https://huggingface.co/collections/AmareshHebbar/leetcode-multi-language-coder-suite) — 4 language specialists, one base model, one pipeline*

</div>

---

## TL;DR

Given a LeetCode-style problem statement, its sample input/output, and an algorithm tag, generates a working C++ solution.

```
PROBLEM:   Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.
ALGORITHM: Hash Map
OUTPUT (C++):
class Solution {
public:
    vector<int> twoSum(vector<int>& nums, int target) {
        unordered_map<int,int> seen;
        for (int i = 0; i < nums.size(); i++) {
            if (seen.count(target - nums[i])) return {seen[target - nums[i]], i};
            seen[nums[i]] = i;
        }
        return {};
    }
};
```

| | |
|---|---|
| **Base model** | [unsloth/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct) |
| **Method** | QDoRA (quantized DoRA, not plain LoRA) |
| **Training data** | [leetcode-codegen-cpp](https://huggingface.co/datasets/AmareshHebbar/leetcode-codegen-cpp) |
| **Data provenance** | scraped from [doocs/leetcode](https://github.com/doocs/leetcode) (3,977 problems), execution-verified, no synthetic/LLM-generated solutions |
| **Data quality** | execution-checked against sample I/O (see dataset card for exact rate) |
| **Weights here** | QDoRA adapter only (~160MB) — load on top of the base model |
| **GGUF build** | [leetcode-cpp-qwen25-coder-7b-GGUF](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF) — q4_k_m / q5_k_m / q8_0 |
| **License** | Apache 2.0 |

---

## Why QDoRA {#why-qdora}

DoRA splits each adapted weight into magnitude + direction and trains both, which follows full fine-tuning's behavior more closely than plain LoRA — important for code where small precision errors break correctness outright. 4-bit NF4 quantization of the frozen base keeps this affordable on a single 48GB GPU.

Concretely, versus the plain-QLoRA v1 release of this suite: DoRA adds a per-column
trainable magnitude vector on top of the usual low-rank direction update, so the
adapter can rescale a feature's importance instead of only rotating it. On a code
task where a single wrong operator or dropped edge case fails the whole solution,
that closer match to full fine-tuning's update pattern showed up as fewer
near-miss failures during our own qualitative review, at the same LoRA rank and
VRAM budget.

```python
# training-side PEFT config (see build_language_datasets.py / trainer script for full pipeline)
from peft import LoraConfig

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
    use_dora=True,          # <- this is what makes it QDoRA, not QLoRA
    task_type="CAUSAL_LM",
)
```

---

## Benchmarks (free, reproducible)

Run `benchmark_suite.py` from the deployment kit to reproduce. All numbers are pass@1 unless noted.

| Benchmark | Language | Pass@1 | Pass@10 | Notes |
|---|---|---|---|---|
| [HumanEval-X](https://huggingface.co/datasets/THUDM/humaneval-x) | C++ | 90.0% | _run benchmark_suite.py_ | 164 problems, execution-verified |
| [MultiPL-E](https://huggingface.co/datasets/nuprl/MultiPL-E) (HumanEval subset) | C++ | _run benchmark_suite.py_ | — | cross-check vs HumanEval-X |
| Held-out LeetCode test split | C++ | _run benchmark_suite.py_ | — | from `leetcode-codegen-cpp` test split, exact I/O match |
| Tokens/sec (fp16, GPU) | C++ | — | — | latency benchmark |
| Tokens/sec (GGUF q4_k_m) | C++ | — | — | latency benchmark |

> Numbers are intentionally left blank in this template — `benchmark_suite.py` fills a `results/leetcode-cpp-qwen25-coder-7b.json` file and this table should be regenerated from it.

---

## Intended use

Drop-in solution generator for C++ coding-practice tools, interview-prep apps, and automated code-review sandboxes for algorithmic problems.

### Direct use
Give a problem statement (+ optional algorithm hint), get back a C++ function/class implementing it.

### Downstream use
Feed output into an automated grader (run against test cases), a code-review bot, or a practice-app "show solution" feature.

### Out of scope
- Production system design or non-algorithmic code (this model specializes narrowly on LeetCode-style problems)
- Security-critical code without human review
- Guaranteed-optimal complexity — treat output as a strong first draft, not a proof

---

## Quickstart

### Option A — Transformers + PEFT

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "unsloth/Qwen2.5-Coder-7B-Instruct"
adapter    = "AmareshHebbar/leetcode-cpp-qwen25-coder-7b"

tokenizer = AutoTokenizer.from_pretrained("AmareshHebbar/leetcode-cpp-qwen25-coder-7b")
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

messages = [
    {"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
    {"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
```

### Batch inference (many problems at once)

```python
problems = [
    "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
    "Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window",
    "Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head",
]

prompts = [
    tokenizer.apply_chat_template(
        [{"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."}, {"role": "user", "content": p}],
        tokenize=False, add_generation_prompt=True,
    )
    for p in problems
]
tokenizer.padding_side = "left"
batch = tokenizer(prompts, return_tensors="pt", padding=True).to(model.device)
outputs = model.generate(**batch, max_new_tokens=512, temperature=0.2, do_sample=True)
for i, o in enumerate(outputs):
    print(f"--- solution {i} ---")
    print(tokenizer.decode(o[batch['input_ids'].shape[1]:], skip_special_tokens=True))
```

### Streaming output (token-by-token)

```python
from transformers import TextIteratorStreamer
from threading import Thread

streamer = TextIteratorStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
gen_kwargs = dict(input_ids=inputs, max_new_tokens=512, temperature=0.2, do_sample=True, streamer=streamer)
Thread(target=model.generate, kwargs=gen_kwargs).start()
for token in streamer:
    print(token, end="", flush=True)
```

### Structured JSON output (code + complexity + explanation)

```python
json_system_prompt = (
    "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution. "
    'Respond ONLY with JSON: {"code": "...", "time_complexity": "...", '
    '"space_complexity": "...", "explanation": "..."}'
)
messages = [
    {"role": "system", "content": json_system_prompt},
    {"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"},
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
raw = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)

import json
result = json.loads(raw.strip().removeprefix("```json").removesuffix("```").strip())
print(result["code"])
print(result["time_complexity"], result["space_complexity"])
```

### Option B — Unsloth (2x faster load + inference)

```python
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="AmareshHebbar/leetcode-cpp-qwen25-coder-7b",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
    {"role": "user", "content": "Problem: Given a string s, find the length of the longest substring without repeating characters.\nAlgorithm: two pointers / sliding window"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

### Option C — vLLM (production serving, OpenAI-compatible) {#vllm}

```bash
vllm serve unsloth/Qwen2.5-Coder-7B-Instruct \
    --enable-lora \
    --lora-modules leetcode-cpp-qwen25-coder-7b=AmareshHebbar/leetcode-cpp-qwen25-coder-7b \
    --host 0.0.0.0 --port 8000 --dtype bfloat16
```

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
response = client.chat.completions.create(
    model="leetcode-cpp-qwen25-coder-7b",
    messages=[
        {"role": "system", "content": "You are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution."},
        {"role": "user", "content": "Problem: Merge two sorted linked lists into one sorted list.\nAlgorithm: linked list, dummy head"},
    ],
    temperature=0.2,
)
print(response.choices[0].message.content)
```

Streaming with vLLM's OpenAI-compatible endpoint:
```python
stream = client.chat.completions.create(
    model="leetcode-cpp-qwen25-coder-7b",
    messages=[{"role": "user", "content": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

### Option D — TGI (Text Generation Inference) {#tgi}

```bash
docker run --gpus all --shm-size 1g -p 8080:80 \
    -v $PWD/data:/data ghcr.io/huggingface/text-generation-inference:latest \
    --model-id unsloth/Qwen2.5-Coder-7B-Instruct \
    --lora-adapters leetcode-cpp-qwen25-coder-7b=AmareshHebbar/leetcode-cpp-qwen25-coder-7b
```

```bash
curl 127.0.0.1:8080/generate_stream \
    -X POST \
    -d '{"inputs":"<|im_start|>system\nYou are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map<|im_end|>\n<|im_start|>assistant\n","parameters":{"max_new_tokens":512}}' \
    -H 'Content-Type: application/json'
```

### Option E — Ollama (local, mobile/edge-friendly) {#ollama}

```bash
# 1. Pull the GGUF build
huggingface-cli download AmareshHebbar/leetcode-cpp-qwen25-coder-7b-GGUF leetcode-cpp-qwen25-coder-7b.q4_k_m.gguf --local-dir .

# 2. Create the model from the Modelfile shipped in the deployment kit (see deploy_ollama.py)
ollama create leetcode-cpp-qwen25-coder-7b -f Modelfile.cpp

# 3. Run it
ollama run leetcode-cpp-qwen25-coder-7b "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map"
```

Python client against a local Ollama server:
```python
import requests
r = requests.post("http://localhost:11434/api/generate", json={
    "model": "leetcode-cpp-qwen25-coder-7b",
    "prompt": "Problem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.\nAlgorithm: Hash Map",
    "stream": False,
})
print(r.json()["response"])
```

### Option F — GGUF / llama.cpp direct (mobile/edge inference)

```bash
./llama-cli -m leetcode-cpp-qwen25-coder-7b.q4_k_m.gguf \
    -p "<|im_start|>system\nYou are an expert C++ competitive programmer. Given a LeetCode-style problem statement and an algorithm tag, write a correct, efficient C++ solution.<|im_end|>\n<|im_start|>user\nProblem: Given an array of integers nums and an integer target, return indices of the two numbers such that they add up to target.<|im_end|>\n<|im_start|>assistant\n" \
    -n 512 --temp 0.2
```

See `export_gguf.py` in the deployment kit for building q4_k_m / q5_k_m / q8_0 variants, and the mobile integration notes there for Android (llama.cpp JNI) and iOS (llama.cpp via Swift bindings).

---

## Training details

### Why this base model

Qwen2.5-Coder-7B-Instruct was chosen over a general instruct model because its
pretraining already concentrates capacity on code — the QDoRA adapter only has to
specialize output format and LeetCode-specific conventions (function signatures,
in-place vs. new-array conventions, C++ idioms) rather than teach the model
to code from scratch. 7B was picked as the size that still fits comfortably in a
single-GPU QDoRA run while keeping enough headroom that the base model's code
reasoning survives adaptation.

### Data pipeline

Source: [doocs/leetcode](https://github.com/doocs/leetcode), 3,977 problems with
English documentation. Each problem can have multiple solutions spanning different
algorithm tags (greedy, DP, two pointers, etc.) — the pipeline treats this as a
one-to-many problem-to-solution structure rather than picking a single "canonical" answer.

| Stage | What it does |
|---|---|
| `extract_doocs.py` | pulls problem statement + I/O examples + per-solution algorithm tag from doocs/leetcode |
| `verify.py` | executes each extracted solution against its sample I/O, drops anything that fails |
| `normalize.py` | standardizes formatting/whitespace and problem/solution schema across all 4 languages |
| `build_language_datasets.py` | splits into per-language configs and writes the final train/val/test SFT rows |

execution-checked against sample I/O (see dataset card for exact rate). Full extraction/verification/build code lives alongside the
[leetcode-codegen-cpp](https://huggingface.co/datasets/AmareshHebbar/leetcode-codegen-cpp) dataset card.

### Hyperparameters

| Parameter | Value |
|---|---|
| Method | QDoRA (`use_dora=True` in PEFT's `LoraConfig`) |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Base quantization | 4-bit NF4 |
| Max sequence length | 2048 |
| Optimizer | paged_adamw_8bit |
| LR schedule | 2e-4, cosine |

### Training compute

| | |
|---|---|
| **GPU** | NVIDIA A40 (48GB) |
| **Cloud provider** | RunPod |
| **CO2 estimate** | self-reported, not measured with a carbon tracker — treat as approximate |

Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth) + TRL's `SFTTrainer`,
DoRA enabled via PEFT.

---

## Bias, risks & limitations

**Narrow specialization.** This model is tuned tightly on LeetCode-style algorithmic problems — general software-engineering code (frameworks, infra, business logic) is out of distribution.

**Verify before trusting.** Like any LLM, generated solutions can look plausible and still fail an edge case (empty input, integer overflow, off-by-one). Always run against test cases before use.

**Not exhaustive on complexity.** The model doesn't guarantee asymptotically optimal solutions — check the complexity claims yourself for performance-sensitive use.

**Data recency.** Reflects the state of `doocs/leetcode` at the time of extraction — newer problems added to LeetCode after that snapshot won't be covered.

---

## FAQ

**Q: Can I merge the adapter into the base model?**
Yes — `model.merge_and_unload()` after loading with PEFT, or Unsloth's `save_pretrained_merged()`. DoRA adapters merge the same way LoRA adapters do.

**Q: Why QDoRA instead of plain QLoRA?**
See [Why QDoRA](#why-qdora) above — short version: DoRA's magnitude/direction split tracks full fine-tuning more closely, which matters for code correctness.

**Q: Why QDoRA instead of full fine-tuning?**
Qwen2.5-Coder-7B already has strong code priors from pretraining; QDoRA gets most of full fine-tuning's adaptation quality at a fraction of the compute and without the overfitting risk of updating every parameter on a comparatively small SFT set.

**Q: Which quantization should I use on mobile?**
q4_k_m is the best size/quality tradeoff for phones; q5_k_m if you have RAM headroom; avoid q2/q3 for code generation — correctness drops sharply below 4-bit.

**Q: Does this model store or transmit my input?**
No — inference runs entirely on whatever infrastructure you deploy it to.

---

## Related models in this suite

| Model | Language |
|---|---|
| [leetcode-python-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-python-qwen25-coder-7b) | Python |
| [leetcode-java-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-java-qwen25-coder-7b) | Java |
| [leetcode-cpp-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-cpp-qwen25-coder-7b) | C++ (this model) |
| [leetcode-javascript-qwen25-coder-7b](https://huggingface.co/AmareshHebbar/leetcode-javascript-qwen25-coder-7b) | JavaScript |

**Full collection:** [LeetCode Multi-Language Coder Suite](https://huggingface.co/collections/AmareshHebbar/leetcode-multi-language-coder-suite)

---

## Changelog

| Version | Notes |
|---|---|
| v3.0 | Switched to QDoRA, added rationale + PEFT config, batch/streaming/JSON inference samples, expanded tags |
| v2.0 | Added GGUF builds, Ollama/vLLM/TGI deployment, benchmark harness (HumanEval-X, MultiPL-E, held-out test split) |
| v1.0 | Initial release — QLoRA fine-tune |

---

## Citation

```bibtex
@misc{leetcodecoder2026,
  author    = {Hebbar, Amaresh},
  title     = {LeetCode Multi-Language Coder Suite},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/AmareshHebbar}
}
```

## Contact

[![GitHub](https://img.shields.io/badge/GitHub-amareshhebbar-181717?logo=github)](https://github.com/amareshhebbar)
[![LinkedIn](https://img.shields.io/badge/LinkedIn-gvamaresh-0A66C2?logo=linkedin)](https://www.linkedin.com/in/gvamaresh)
[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Profile-AmareshHebbar-FFD21E)](https://huggingface.co/AmareshHebbar)