Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,8 @@ Unlike standard distillation methods that rely on unstructured Chain-of-Thought
|
|
| 25 |
|
| 26 |
📄 **Paper:** [Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL](https://arxiv.org/abs/2512.17053)
|
| 27 |
|
|
|
|
|
|
|
| 28 |
## Performance
|
| 29 |
|
| 30 |
On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy (EX) of 45.0%**, outperforming standard unstructured CoT distillation baselines by **8.1 points**.
|
|
@@ -36,6 +38,7 @@ On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy
|
|
| 36 |
| FN-Gold Baseline | No Reasoning (SQL Only) | 34.3% |
|
| 37 |
| Base Student (Zero-shot) | None | 17.0% |
|
| 38 |
|
|
|
|
| 39 |
## Methodology
|
| 40 |
|
| 41 |
The model was trained on a curated dataset of **1,000 samples** generated by GPT-4o. The training data consists of:
|
|
@@ -44,10 +47,12 @@ The model was trained on a curated dataset of **1,000 samples** generated by GPT
|
|
| 44 |
|
| 45 |
By forcing the model to explicitly plan the query execution (e.g., "Scan Table", "Filter by...", "Join with..."), the model learns the logical structure of SQL generation rather than just memorizing patterns.
|
| 46 |
|
|
|
|
| 47 |
## Usage
|
| 48 |
|
| 49 |
You can use this model with the `transformers` library. It expects the input to be formatted with a specific system prompt or structure if you want to elicit the query plan.
|
| 50 |
|
|
|
|
| 51 |
```python
|
| 52 |
import torch
|
| 53 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -65,8 +70,7 @@ inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
|
|
| 65 |
outputs = model.generate(**inputs, max_new_tokens=1200)
|
| 66 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 67 |
```
|
| 68 |
-
|
| 69 |
-
|
| 70 |
## Intended Use
|
| 71 |
|
| 72 |
Struct-SQL-4B is intended for **research and academic use** in tasks involving **Text-to-SQL generation** and **semantic parsing over relational databases**. The model is particularly suited for studying:
|
|
@@ -79,14 +83,12 @@ Struct-SQL-4B is intended for **research and academic use** in tasks involving *
|
|
| 79 |
The model is not optimized for direct deployment in production database systems without additional validation and safety constraints.
|
| 80 |
|
| 81 |
---
|
| 82 |
-
|
| 83 |
## Limitations
|
| 84 |
|
| 85 |
- Evaluation is confined to the SQLite-based BIRD benchmark
|
| 86 |
- The model may generate logically plausible but incorrect SQL for highly complex multi-hop queries
|
| 87 |
|
| 88 |
---
|
| 89 |
-
|
| 90 |
## Citation
|
| 91 |
|
| 92 |
```bibtex
|
|
|
|
| 25 |
|
| 26 |
📄 **Paper:** [Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQL](https://arxiv.org/abs/2512.17053)
|
| 27 |
|
| 28 |
+
---
|
| 29 |
+
|
| 30 |
## Performance
|
| 31 |
|
| 32 |
On the **BIRD mini-dev** benchmark, Struct-SQL achieves an **Execution Accuracy (EX) of 45.0%**, outperforming standard unstructured CoT distillation baselines by **8.1 points**.
|
|
|
|
| 38 |
| FN-Gold Baseline | No Reasoning (SQL Only) | 34.3% |
|
| 39 |
| Base Student (Zero-shot) | None | 17.0% |
|
| 40 |
|
| 41 |
+
---
|
| 42 |
## Methodology
|
| 43 |
|
| 44 |
The model was trained on a curated dataset of **1,000 samples** generated by GPT-4o. The training data consists of:
|
|
|
|
| 47 |
|
| 48 |
By forcing the model to explicitly plan the query execution (e.g., "Scan Table", "Filter by...", "Join with..."), the model learns the logical structure of SQL generation rather than just memorizing patterns.
|
| 49 |
|
| 50 |
+
---
|
| 51 |
## Usage
|
| 52 |
|
| 53 |
You can use this model with the `transformers` library. It expects the input to be formatted with a specific system prompt or structure if you want to elicit the query plan.
|
| 54 |
|
| 55 |
+
---
|
| 56 |
```python
|
| 57 |
import torch
|
| 58 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 70 |
outputs = model.generate(**inputs, max_new_tokens=1200)
|
| 71 |
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 72 |
```
|
| 73 |
+
---
|
|
|
|
| 74 |
## Intended Use
|
| 75 |
|
| 76 |
Struct-SQL-4B is intended for **research and academic use** in tasks involving **Text-to-SQL generation** and **semantic parsing over relational databases**. The model is particularly suited for studying:
|
|
|
|
| 83 |
The model is not optimized for direct deployment in production database systems without additional validation and safety constraints.
|
| 84 |
|
| 85 |
---
|
|
|
|
| 86 |
## Limitations
|
| 87 |
|
| 88 |
- Evaluation is confined to the SQLite-based BIRD benchmark
|
| 89 |
- The model may generate logically plausible but incorrect SQL for highly complex multi-hop queries
|
| 90 |
|
| 91 |
---
|
|
|
|
| 92 |
## Citation
|
| 93 |
|
| 94 |
```bibtex
|