Update README.md: add training details, fix AutoModelForCausalLM, update model overview

README.md

@@ -24,96 +24,109 @@ model-index:
   results:
   - task:
       type: text-generation
-      name: Code Generation
+      name: CodeGeneration
     dataset:
       name: HumanEval
       type: openai/openai_humaneval
     metrics:
     - type: pass_at_1
-      value: 50.0
+      value: 50
       verified: false
     - type: pass_at_10
-      value: 75.0
+      value: 75
       verified: false
   - task:
       type: text-generation
-      name: Python Code Generation
+      name: PythonCodeGeneration
     dataset:
       name: MBPP
       type: abdshhayan/MBPP
     metrics:
     - type: pass_at_1
-      value: 55.0
+      value: 55
       verified: false
 ---
 
-# 🤖 amk-coder-v2
+# 🤖 amk-coder-v2 — Myanmar Coding Agent
 
-Myanmar Coding Assistant - Fine-tuned from **Qwen2.5-Coder-1.5B**
+Myanmar Coding Assistant — Fine-tuned from **Qwen2.5-Coder-1.5B** using **LoRA (PEFT)**
 
-[![Model](https://img.shields.io/badge/Model-Size-2B-blue)](https://huggingface.co/amkyawdev/amk-coder-v2)
-[![License](https://img.shields.io/badge/License-Apache--2.0-green)](LICENSE)
-[![HuggingFace](https://img.shields.io/badge/🤗-HuggingFace-orange)](https://huggingface.co/amkyawdev/amk-coder-v2)
+![Model](https://img.shields.io/badge/Model-Size-2B-blue)
+![License](https://img.shields.io/badge/License-Apache--2.0-green)
+![HuggingFace](https://img.shields.io/badge/-HuggingFace-orange)
 
 ---
 
 ## 📋 Table of Contents
 
-- [Features](#-features)
-- [Model Details](#-model-details)
-- [Quick Start](#-quick-start)
-- [Usage Examples](#-usage-examples)
-- [API Endpoints](#-api-endpoints)
-- [Deployment](#-deployment)
-- [Limitations](#-limitations)
-- [License](#-license)
+- [Model Overview](#model-overview)
+- [Training Details](#training-details)
+- [Quick Start](#quick-start)
+- [Usage Examples](#usage-examples)
+- [API Deployment](#api-deployment)
+- [Limitations](#limitations)
+- [License](#license)
 
 ---
 
-## ✨ Features
+## Model Overview
 
-| Feature | Description |
-|---------|-------------|
-| 🇲🇲 **Myanmar Support** | Full support for Myanmar Unicode text |
-| 💻 **Code Generation** | Python, JavaScript, C++, Java, and more |
-| 🐛 **Debugging** | Bug detection and fixes |
-| 📖 **Code Explanation** | Line-by-line explanations |
-| 🔍 **Web Search** | Integration for latest documentation |
-| 🌐 **Streaming** | Real-time response streaming |
-| 🎨 **Beautiful UI** | Modern chat interface |
-
----
-
-## 📊 Model Details
+**amk-coder-v2** is a Myanmar-localized coding assistant fine-tuned from **Qwen2.5-Coder-1.5B** using LoRA (PEFT) technique.
 
 | Attribute | Value |
-|-----------|-------|
+|---|---|
 | **Base Model** | Qwen2.5-Coder-1.5B |
 | **Parameters** | 2B (2,000M) |
 | **Architecture** | Qwen2ForCausalLM |
-| **Hidden Size** | 1536 |
-| **Layers** | 28 |
-| **Attention Heads** | 12 |
-| **Vocab Size** | 151,936 |
+| **Training Method** | LoRA (PEFT) fine-tuning |
+| **Dataset** | [amkyawdev/mm-llm-coder-agent-dataset](https://huggingface.co/datasets/amkyawdev/mm-llm-coder-agent-dataset) (4M rows) |
 | **Context Length** | 32,768 tokens |
-| **Tensor Type** | BF16 |
-| **Format** | Safetensors |
+| **Format** | Safetensors (BF16) |
 | **License** | Apache-2.0 |
+| **Languages** | Burmese + English |
+
+### Features
+
+| Feature | Description |
+|---|---|
+| 🇲🇲 **Myanmar Support** | Full support for Myanmar Unicode text |
+| 💻 **Code Generation** | Python, JavaScript, C++, Java, and more |
+| 🐛 **Debugging** | Bug detection and fixes |
+| 📖 **Code Explanation** | Line-by-line explanations |
 
-### Training Details
+---
+
+## Training Details
 
 | Parameter | Value |
-|-----------|-------|
+|---|---|
 | **Framework** | Transformers + PEFT |
 | **Training Method** | LoRA fine-tuning |
-| **Learning Rate** | 2e-4 |
+| **Target Modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| **Optimizer** | paged_adamw_8bit |
+| **Learning Rate** | 3e-5 |
 | **Epochs** | 3 |
 | **Batch Size** | 8 |
 | **Max Length** | 2048 |
+| **Precision** | FP16 mixed |
+| **Hardware** | Kaggle Dual T4 GPU |
+| **Training Time** | ~3-5 hrs |
+
+### Chat Template (ChatML)
+
+```
+<|im_start|>system
+You are an expert Myanmar AI coding agent with tool access.<|im_end|>
+<|im_start|>user
+{Instruction}
+Tools available: {Tools}<|im_end|>
+<|im_start|>assistant
+Thought & Code:
+```
 
 ---
 
-## 🚀 Quick Start
+## Quick Start
 
 ### Using Transformers (Python)
 
@@ -123,7 +136,7 @@ from transformers import pipeline
 
 pipe = pipeline("text-generation", model="amkyawdev/amk-coder-v2")
 messages = [
-    {"role": "user", "content": "Python နဲ့ list comprehension ရေးပါ"}
+    {"role": "user", "content": "Python function တစ်ခုရေးပါ။ list comprehension နဲ့ sorting လုပ်ပေးပါ။"}
 ]
 result = pipe(messages, max_new_tokens=512, temperature=0.2)
 print(result[0]['generated_text'])
@@ -143,7 +156,13 @@ messages = [
     {"role": "system", "content": "You are a helpful coding assistant."},
     {"role": "user", "content": "Write a Python function to reverse a string"}
 ]
-inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
 outputs = model.generate(inputs, max_new_tokens=512, temperature=0.2)
 response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
 print(response)
@@ -191,12 +210,11 @@ curl -X POST "http://localhost:30000/v1/chat/completions" \
 
 ---
 
-## 💡 Usage Examples
+## Usage Examples
 
 ### 🇲🇲 Myanmar Prompts
 
 ```python
-# Code generation in Myanmar
 messages = [
     {"role": "user", "content": "Python function တစ်ခုရေးပါ။ ဂဏန်းတွေကို sorting လုပ်ပေးပါ။"}
 ]
@@ -223,7 +241,7 @@ messages = [
 
 ---
 
-## 🔗 API Endpoints
+## API Deployment
 
 ### Backend Server
 
@@ -237,7 +255,7 @@ uvicorn app.main:app --host 0.0.0.0 --port 8000
 ### Endpoints
 
 | Method | Endpoint | Description |
-|--------|----------|-------------|
+|---|---|---|
 | GET | `/` | Health check |
 | GET | `/health` | Service health status |
 | POST | `/chat` | Streaming chat (SSE) |
@@ -258,11 +276,7 @@ curl -X POST "http://localhost:8000/chat" \
   }'
 ```
 
----
-
-## 🐳 Deployment
-
-### Docker
+### Docker Deployment
 
 ```bash
 # Using Docker Model Runner
@@ -277,14 +291,6 @@ docker run --gpus all \
   --model amkyawdev/amk-coder-v2
 ```
 
-### Google Colab
-
-[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/amkyawdev/amk-coder-v2)
-
-### Kaggle
-
-[![Kaggle](https://img.shields.io/badge/Kaggle-Notebook-orange)](https://www.kaggle.com)
-
 ---
 
 ## ⚠️ Limitations
@@ -301,7 +307,7 @@ docker run --gpus all \
 
 - [Qwen2.5-Coder Documentation](https://qwenlm.github.io/blog/Qwen2.5-Coder/)
 - [Transformers Library](https://huggingface.co/docs/transformers)
-- [HuggingFace Hub](https://huggingface.co/docs/hub)
+- [HuggingFace Hub](https://huggingface.co/amkyawdev/amk-coder-v2)
 
 ---
 
@@ -315,7 +321,7 @@ docker run --gpus all \
 
 ## 📝 License
 
-Apache License 2.0 - See [LICENSE](LICENSE) file for details.
+Apache License 2.0 - See LICENSE file for details.
 
 ---
 
@@ -327,4 +333,4 @@ Apache License 2.0 - See [LICENSE](LICENSE) file for details.
 
 ---
 
-*Made with ❤️ for Myanmar Developers*
+*Made with ❤️ for Myanmar Developers*