spcv/qwen2.5_coder_text2sql_onnx

🧠 Qwen2.5-Coder-1.5B-Instruct Text-to-SQL (ONNX GenAI INT4)

This repository hosts an optimized, fine-tuned Text-to-SQL Small Language Model (SLM) based on Qwen/Qwen2.5-Coder-1.5B-Instruct.

Fine-tuned on the trl-lab/SQaLe-text-to-SQL dataset using QLoRA and exported to ONNX Runtime GenAI (INT4) for ultra-low latency, CPU/edge execution with negligible RAM and VRAM footprint.

---

📌 Model Highlights

• Base Architecture: Qwen2.5-Coder-1.5B-Instruct
• Fine-Tuning Technique: QLoRA (Rank r=64, Alpha 128, Targets: q, k, v, o, gate, up, down projections)
• Quantization & Format: ONNX Runtime GenAI (INT4 / DirectML / CPU / CUDA compatible)
• Model Size: ~980 MB (INT4 quantized model.onnx.data)
• Primary Use Case: Precise schema-aware Natural Language to SQL query translation for enterprise databases, analytical engines, and autonomous multi-agent pipelines.

---

🛠️ Quickstart & Inference

1. Installation

pip install onnxruntime-genai huggingface_hub

2. Download and Run Inference

import os
import onnxruntime_genai as og
from huggingface_hub import snapshot_download

# 1. Download model from Hugging Face Hub
REPO_ID = "spcv/qwen2.5_coder_text2sql_onnx"
model_dir = snapshot_download(repo_id=REPO_ID)

# 2. Load the ONNX model and tokenizer
model = og.Model(model_dir)
tokenizer = og.Tokenizer(model)

# 3. Define the Database Schema & Question
schema = """
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    email VARCHAR(100),
    created_at TIMESTAMP
);

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT REFERENCES customers(customer_id),
    order_date DATE,
    total_amount DECIMAL(10, 2),
    status VARCHAR(20)
);
"""

question = "Find the total amount spent by customer with email 'jane.doe@example.com' on completed orders."

# 4. Construct Prompt using the Qwen ChatML Template
system_prompt = (
    "You are an expert SQL query writer. Follow these rules strictly:\n"
    "1. Only use tables and columns that exist in the provided schema.\n"
    "2. Use correlated subqueries or JOINs when a value must be derived from another table.\n"
    "3. Use IS NULL / IS NOT NULL for null checks, never != '' or = ''.\n"
    "4. Use the correct aggregation: SUM for totals, COUNT for row counts, AVG for averages.\n"
    "5. Write syntactically valid SQL: WHERE must come after all JOINs.\n"
    "6. Return only the SQL query with no explanation or markdown."
)

user_content = f"### Database Schema\n{schema.strip()}\n\n### Question\n{question}\n\n### SQL Query"

prompt = (
    f"<|im_start|>system\n{system_prompt}<|im_end|>\n"
    f"<|im_start|>user\n{user_content}<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

# 5. Tokenize and Generate
tokens = tokenizer.encode(prompt)
params = og.GeneratorParams(model)
params.set_search_options(max_length=512, temperature=0.1, top_p=0.9)
params.input_ids = tokens

generator = og.Generator(model, params)
generated_tokens = []

while not generator.is_done():
    generator.compute_logits()
    generator.generate_next_token()
    new_token = generator.get_next_tokens()[0]
    generated_tokens.append(new_token)

output_sql = tokenizer.decode(generated_tokens)
print("Generated SQL:\n", output_sql.strip())

---

🎯 Prompt & Chat Template Structure

The model follows standard ChatML format with structured instructions:

<|im_start|>system
You are an expert SQL query writer. Follow these rules strictly:
1. Only use tables and columns that exist in the provided schema.
2. Use correlated subqueries or JOINs when a value must be derived from another table.
3. Use IS NULL / IS NOT NULL for null checks, never != '' or = ''.
4. Use the correct aggregation: SUM for totals, COUNT for row counts, AVG for averages.
5. Write syntactically valid SQL: WHERE must come after all JOINs.
6. Return only the SQL query with no explanation or markdown.<|im_end|>
<|im_start|>user
### Database Schema
[DDL / Schema definition]

### Question
[User Question in Natural Language]

### SQL Query<|im_end|>
<|im_start|>assistant

---

🏋️ Training & Fine-Tuning Details

Hyperparameters

Parameter	Value
Base Model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Dataset	trl-lab/SQaLe-text-to-SQL
Training Framework	Hugging Face trl (SFTTrainer) + peft
LoRA Rank ($r$)	64
LoRA Alpha ($\alpha$)	128
LoRA Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning Rate	5e-5 (Cosine schedule, 5% warmup)
Precision	bfloat16 / NF4 4-bit base loading
Export Toolchain	onnxruntime-genai.models.builder (-p int4, -e cpu/cuda)

---

📊 Evaluation & Capabilities

• Single & Multi-table JOINs: Correctly resolves foreign keys and table references.
• Aggregations & Filtering: Accurately computes SUM, COUNT, AVG, GROUP BY, and HAVING clauses.
• Subqueries & CTEs: Handles nested filtering and window functions where supported.
• Dialect Support: Standard ANSI SQL / SQLite / PostgreSQL / MySQL compliant syntax.

---

📄 License & Attribution

• Base model licensed under Apache 2.0 by the Qwen Team / Alibaba Cloud.
• Distributed as part of the SLMAgents multi-agent local intelligence suite.