Update README.md

README.md

@@ -1,5 +1,133 @@
 ---
-license: apache-2.0
+base_model: unsloth/granite-4.0-350m-unsloth-bnb-4bit
+library_name: peft
+pipeline_tag: text-generation
 tags:
+- base_model:adapter:unsloth/granite-4.0-350m-unsloth-bnb-4bit
+- lora
+- sft
+- transformers
+- trl
 - unsloth
+- text-to-sql
+license: apache-2.0
+datasets:
+- b-mc2/sql-create-context
+---
+
+# Model Card for granite_sql (checkpoint-6000)
+
+LoRA adapter fine-tuning IBM Granite 4.0 350M for text-to-SQL generation: given a
+`CREATE TABLE` schema and a natural-language question, generate the corresponding SQL query.
+
+## Model Details
+
+### Model Description
+
+This checkpoint (step 6000 of 6432, epoch 2.80/3) is the best-performing checkpoint
+of the run — lowest eval loss among all saved checkpoints (see Results).
+
+- **Model type:** LoRA adapter (PEFT) on a causal LM
+- **Language(s):** SQL (generation), English (instructions)
+- **License:** Apache 2.0 (inherited from base model)
+- **Finetuned from model:** [unsloth/granite-4.0-350m-unsloth-bnb-4bit](https://huggingface.co/unsloth/granite-4.0-350m-unsloth-bnb-4bit) (IBM Granite 4.0, 350M, GraniteMoeHybrid)
+
+## How to Get Started with the Model
+
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="/workspace/outputs/granite_sql_train/checkpoint-6000",
+    load_in_4bit=False,
+    device_map="cuda:0",
+)
+FastLanguageModel.for_inference(model)
+
+instruction = """Generate ONLY the SQL query for the following database.
+
+Do not explain your answer.
+Do not include markdown.
+Do not include any additional text.
+
+Schema:
+{schema}
+
+Question:
+{question}
+
+SQL:"""
+
+messages = [{"role": "user", "content": [{"type": "text", "text": instruction.format(schema=schema, question=question)}]}]
+input_text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+inputs = tokenizer(input_text, add_special_tokens=False, return_tensors="pt").to("cuda")
+output = model.generate(**inputs, max_new_tokens=256, use_cache=True, temperature=0.7, top_p=0.8, top_k=20)
+```
+
+## Training Details
+
+### Training Data
+
+[b-mc2/sql-create-context](https://huggingface.co/datasets/b-mc2/sql-create-context) —
+schema + natural-language-question + SQL-answer triples. Split via
+`train_test_split(test_size=10000, seed=42, shuffle=True)`: remaining rows for train, 10,000 held out for eval.
+
+### Training Procedure
+
+LoRA (r=16, alpha=32, dropout=0, bias=none, no rslora) applied to
+`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, shared_mlp.input_linear, shared_mlp.output_linear`,
+trained with Unsloth + TRL SFTTrainer.
+
+#### Training Hyperparameters
+
+- **Training regime:** bf16 mixed precision
+- Epochs: 3 | Per-device train batch size: 32 | Per-device eval batch size: 64 | Grad accumulation: 1
+- Learning rate: 2e-4, linear schedule, 50 warmup steps
+- Optimizer: adamw_8bit, weight decay 0.001, max grad norm 1.0
+- Seed: 3407 | Eval every 200 steps | Save every 500 steps
+
+#### Speeds, Sizes, Times
+
+Full run: 6432 steps / 3 epochs, train_runtime ≈ 2567s. This adapter checkpoint: ~26.7MB (`adapter_model.safetensors`).
+
+## Evaluation
+
+### Testing Data & Metrics
+
+10,000-row held-out split of `b-mc2/sql-create-context` (see Training Data), evaluated by SFT eval loss (cross-entropy) every 200 steps.
+
+### Results
+
+| Checkpoint | Step | Eval loss |
+|---|---|---|
+| checkpoint-4000 | 4000 | 0.02830 |
+| checkpoint-5000 | 5000 | 0.02811 |
+| **checkpoint-6000** | **6000** | **0.02673 (best saved)** |
+| checkpoint-6432 (final) | 6432 | ~0.02746 (nearest eval at step 6400) |
+
+Lowest eval_loss observed during training was 0.02630 at step 5200, but no checkpoint was
+saved at that exact step (checkpoints every 500 steps, eval every 200), so checkpoint-6000
+is the closest usable minimum. `load_best_model_at_end` was not enabled, so this checkpoint
+was selected manually by comparing `eval_loss` across saved checkpoints.
+
+## Compute Infrastructure
+
+### Hardware
+
+- 1× NVIDIA A100 40GB
+
+### Software
+
+- Unsloth
+- Transformers
+- PyTorch
+
 ---
+
+## Model Card Authors
+
+- Yian
+
+## Contact
+
+- https://github.com/Siruenyian