README.md

---
language:
- en
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
tags:
- text-to-sql
- sql
- qwen
- qwen2.5-coder
- onnx
- onnxruntime-genai
- int4
- slm
datasets:
- trl-lab/SQaLe-text-to-SQL
pipeline_tag: text-generation
library_name: onnxruntime-genai
---

# 🧠 Qwen2.5-Coder-1.5B-Instruct Text-to-SQL (ONNX GenAI INT4)

This repository hosts an optimized, fine-tuned **Text-to-SQL** Small Language Model (SLM) based on [`Qwen/Qwen2.5-Coder-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct).

Fine-tuned on the [`trl-lab/SQaLe-text-to-SQL`](https://huggingface.co/datasets/trl-lab/SQaLe-text-to-SQL) dataset using QLoRA and exported to **ONNX Runtime GenAI (INT4)** for ultra-low latency, CPU/edge execution with negligible RAM and VRAM footprint.

---

## 📌 Model Highlights

- **Base Architecture**: `Qwen2.5-Coder-1.5B-Instruct`
- **Fine-Tuning Technique**: QLoRA (Rank `r=64`, Alpha `128`, Targets: `q, k, v, o, gate, up, down` projections)
- **Quantization & Format**: ONNX Runtime GenAI (INT4 / DirectML / CPU / CUDA compatible)
- **Model Size**: ~980 MB (INT4 quantized `model.onnx.data`)
- **Primary Use Case**: Precise schema-aware Natural Language to SQL query translation for enterprise databases, analytical engines, and autonomous multi-agent pipelines.

---

## 🛠️ Quickstart & Inference

### 1. Installation

```bash
pip install onnxruntime-genai huggingface_hub
```

### 2. Download and Run Inference

```python
import os
import onnxruntime_genai as og
from huggingface_hub import snapshot_download

# 1. Download model from Hugging Face Hub
REPO_ID = "spcv/qwen2.5_coder_text2sql_onnx"
model_dir = snapshot_download(repo_id=REPO_ID)

# 2. Load the ONNX model and tokenizer
model = og.Model(model_dir)
tokenizer = og.Tokenizer(model)

# 3. Define the Database Schema & Question
schema = """
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    email VARCHAR(100),
    created_at TIMESTAMP
);

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT REFERENCES customers(customer_id),
    order_date DATE,
    total_amount DECIMAL(10, 2),
    status VARCHAR(20)
);
"""

question = "Find the total amount spent by customer with email 'jane.doe@example.com' on completed orders."

# 4. Construct Prompt using the Qwen ChatML Template
system_prompt = (
    "You are an expert SQL query writer. Follow these rules strictly:\n"
    "1. Only use tables and columns that exist in the provided schema.\n"
    "2. Use correlated subqueries or JOINs when a value must be derived from another table.\n"
    "3. Use IS NULL / IS NOT NULL for null checks, never != '' or = ''.\n"
    "4. Use the correct aggregation: SUM for totals, COUNT for row counts, AVG for averages.\n"
    "5. Write syntactically valid SQL: WHERE must come after all JOINs.\n"
    "6. Return only the SQL query with no explanation or markdown."
)

user_content = f"### Database Schema\n{schema.strip()}\n\n### Question\n{question}\n\n### SQL Query"

prompt = (
    f"<|im_start|>system\n{system_prompt}<|im_end|>\n"
    f"<|im_start|>user\n{user_content}<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

# 5. Tokenize and Generate
tokens = tokenizer.encode(prompt)
params = og.GeneratorParams(model)
params.set_search_options(max_length=512, temperature=0.1, top_p=0.9)
params.input_ids = tokens

generator = og.Generator(model, params)
generated_tokens = []

while not generator.is_done():
    generator.compute_logits()
    generator.generate_next_token()
    new_token = generator.get_next_tokens()[0]
    generated_tokens.append(new_token)

output_sql = tokenizer.decode(generated_tokens)
print("Generated SQL:\n", output_sql.strip())
```

---

## 🎯 Prompt & Chat Template Structure

The model follows standard **ChatML** format with structured instructions:

```text
<|im_start|>system
You are an expert SQL query writer. Follow these rules strictly:
1. Only use tables and columns that exist in the provided schema.
2. Use correlated subqueries or JOINs when a value must be derived from another table.
3. Use IS NULL / IS NOT NULL for null checks, never != '' or = ''.
4. Use the correct aggregation: SUM for totals, COUNT for row counts, AVG for averages.
5. Write syntactically valid SQL: WHERE must come after all JOINs.
6. Return only the SQL query with no explanation or markdown.<|im_end|>
<|im_start|>user
### Database Schema
[DDL / Schema definition]

### Question
[User Question in Natural Language]

### SQL Query<|im_end|>
<|im_start|>assistant
```

---

## 🏋️ Training & Fine-Tuning Details

### Hyperparameters
| Parameter | Value |
|---|---|
| **Base Model** | `Qwen/Qwen2.5-Coder-1.5B-Instruct` |
| **Dataset** | `trl-lab/SQaLe-text-to-SQL` |
| **Training Framework** | Hugging Face `trl` (`SFTTrainer`) + `peft` |
| **LoRA Rank ($r$)** | `64` |
| **LoRA Alpha ($\alpha$)** | `128` |
| **LoRA Target Modules** | `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj` |
| **Learning Rate** | `5e-5` (Cosine schedule, 5% warmup) |
| **Precision** | `bfloat16` / `NF4` 4-bit base loading |
| **Export Toolchain** | `onnxruntime-genai.models.builder` (`-p int4`, `-e cpu/cuda`) |

---

## 📊 Evaluation & Capabilities

- **Single & Multi-table JOINs**: Correctly resolves foreign keys and table references.
- **Aggregations & Filtering**: Accurately computes `SUM`, `COUNT`, `AVG`, `GROUP BY`, and `HAVING` clauses.
- **Subqueries & CTEs**: Handles nested filtering and window functions where supported.
- **Dialect Support**: Standard ANSI SQL / SQLite / PostgreSQL / MySQL compliant syntax.

---

## 📄 License & Attribution
- Base model licensed under **Apache 2.0** by the Qwen Team / Alibaba Cloud.
- Distributed as part of the **SLMAgents** multi-agent local intelligence suite.