Update README.md

README.md

@@ -1,3 +1,117 @@
 ---
 license: mit
+base_model: unsloth/Llama-3.2-3B-Instruct
+tags:
+  - text-to-sql
+  - sql
+  - lora
+  - unsloth
+  - llama
+language:
+  - en
 ---
+
+# Llama-3.2-3B Text-to-SQL
+
+A LoRA fine-tune of [Llama-3.2-3B-Instruct](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct)
+for generating SQL queries from a natural language question plus a table schema.
+
+Trained locally on an AMD Radeon RX 9070 XT using [Unsloth](https://github.com/unslothai/unsloth).
+
+## What this model does
+
+Given a table schema and a question in plain English, it returns a SQL query — no
+explanation, no preamble, no alternative approaches. The base instruct model tends to
+either refuse ("I don't have access to your database") or respond with an explanatory
+Python example instead of raw SQL. Fine-tuning fixed that: this model reliably answers
+with bare SQL by default.
+
+## Prompt format
+
+**This model expects the schema to be included in the prompt.** Without one, it will
+guess plausible-sounding table/column names rather than asking for clarification — same
+as any model would.
+
+```
+Given this schema: CREATE TABLE employees (name VARCHAR, salary INTEGER)
+
+Answer this question in SQL: List the names of employees who earn more than 50000
+```
+
+Expected output:
+```sql
+SELECT name FROM employees WHERE salary > 50000
+```
+
+Use the tokenizer's chat template (`apply_chat_template`) with this as a single user
+turn — see the usage example below.
+
+## Usage
+
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "crimson3327/text_to_sql",
+    max_seq_length = 1024,
+)
+FastLanguageModel.for_inference(model)
+
+schema = "CREATE TABLE employees (name VARCHAR, salary INTEGER)"
+question = "List the names of employees who earn more than 50000"
+
+inputs = tokenizer.apply_chat_template(
+    [{"role": "user", "content": f"Given this schema: {schema}\n\nAnswer this question in SQL: {question}"}],
+    tokenize=True, add_generation_prompt=True, return_tensors="pt"
+).to("cuda")
+
+outputs = model.generate(input_ids=inputs, max_new_tokens=150)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training details
+
+- **Base model:** unsloth/Llama-3.2-3B-Instruct
+- **Method:** LoRA, rank 16, alpha 16, all attention + MLP projections targeted
+- **Dataset:** [b-mc2/sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context)
+  (78,577 question/schema/SQL triples), trained on the first 74,000 rows
+- **Held-out eval set:** the remaining ~4,500 rows, never seen during training
+- **Steps:** 1,200 (batch size 2, gradient accumulation 4 — effective batch 8)
+- **Optimizer:** AdamW, linear LR schedule, 2e-4 peak learning rate
+- **Checkpoint selection:** best checkpoint chosen automatically by **eval loss**, not
+  training loss, to avoid shipping an overfit checkpoint
+- **Final eval loss:** 0.559
+
+## Example: base model vs. this fine-tune
+
+Same prompt, no schema given, asked to filter employee data:
+
+**Base Llama-3.2-3B-Instruct** — rewrote the task as a pandas exercise with invented
+sample data, offered a second alternative approach using bitwise operators, no SQL
+produced.
+
+**This model:**
+```sql
+SELECT * FROM employee WHERE salary > 10000 AND Department = 'IT' AND Name = 'John Doe'
+```
+
+## Known limitations
+
+- **Multi-table joins (3+ tables) with aggregation are inconsistent.** Earlier training
+  checkpoints produced invalid SQL (joins placed after `GROUP BY`/`HAVING`) and
+  hallucinated columns on this pattern specifically. The checkpoint published here
+  (trained with the held-out eval split) resolved the specific cases tested, but this
+  remains the weakest area — verify output on complex joins before trusting it blindly.
+- **Ambiguous negation phrasing** (e.g. "not yet shipped") may be interpreted as a
+  literal status string rather than a negated condition (`!=`). Prefer explicit phrasing
+  in questions where this matters.
+- **No schema validation.** The model doesn't verify that referenced columns/tables
+  exist — always pass an accurate schema and review output before executing against a
+  real database.
+- Trained on SQLite-flavored syntax (matching the source dataset); some queries may need
+  adjustment for strict-mode PostgreSQL or other dialects.
+
+## Acknowledgements
+
+Fine-tuned with [Unsloth](https://github.com/unslothai/unsloth). Dataset:
+[b-mc2/sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context).