Ult1-Coding v3 - few-shot in system prompt, 8-module LoRA, training data

README.md

@@ -1,53 +1,52 @@
 ---
 language: en
-library_name: transformers
+library_name: peft
 base_model: Qwen/Qwen2.5-3B-Instruct
 pipeline_tag: text-generation
 tags:
-- qwen
-- qwen2.5
-- 3b
+- base_model:adapter:Qwen/Qwen2.5-3B-Instruct
 - lora
-- coding
-- code
-- software-engineering
+- transformers
 license: apache-2.0
 ---
-
-# Ult1-Coding
-
-**A 3-billion-parameter coding specialist -- master-level software engineer.**
-
-Based on Qwen2.5-3B-Instruct with an embedded **master programmer** system prompt containing few-shot coding demonstrations and a coding-focused LoRA adapter (rank 16, 8 target module types).
-
-## Usage
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1-coding")
-tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1-coding")
-
-messages = [{"role": "user", "content": "Write a Python async web scraper with retry logic"}]
-text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(text, return_tensors="pt")
-outputs = model.generate(**inputs, max_new_tokens=512)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
-
-> The system prompt with few-shot examples is auto-injected by the chat template -- no manual system prompt needed.
-
-## GGUF
-
-Download `Ult1-Coding-Q8_0.gguf` for CPU inference with llama.cpp.
-
-## Training Data
-
-`training_data.json` contains 10 coding Q&A pairs (Python, JavaScript, Rust, SQL, TypeScript, Go). Use with `train.py` on a GPU.
-
-## Details
-
-- **Base**: Qwen2.5-3B-Instruct (3B params)
-- **LoRA**: Rank 16, targets q/k/v/o + gate/up/down projections
-- **Context**: 32,768 tokens
-- **Focus**: Code generation, algorithms, system design, debugging
+
+# Ult1-Coding
+
+**A 3-billion-parameter coding specialist -- master-level software engineer.**
+
+Based on Qwen2.5-3B-Instruct with an embedded **master programmer** system prompt containing few-shot coding demonstrations and a coding-focused LoRA adapter (rank 16, 8 target module types).
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1-coding")
+tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1-coding")
+
+messages = [{"role": "user", "content": "Write a Python async web scraper with retry logic"}]
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model.generate(**inputs, max_new_tokens=512)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+> The system prompt with few-shot examples is auto-injected by the chat template -- no manual system prompt needed.
+
+## GGUF
+
+Download `Ult1-Coding-Q8_0.gguf` for CPU inference with llama.cpp.
+
+## Training Data
+
+`training_data.json` contains 10 coding Q&A pairs (Python, JavaScript, Rust, SQL, TypeScript, Go). Use with `train.py` on a GPU.
+
+## Details
+
+- **Base**: Qwen2.5-3B-Instruct (3B params)
+- **LoRA**: Rank 16, targets q/k/v/o + gate/up/down projections
+- **Context**: 32,768 tokens
+- **Focus**: Code generation, algorithms, system design, debugging
+### Framework versions
+
+- PEFT 0.19.1

adapter_config.json

@@ -31,12 +31,12 @@
   "revision": null,
   "target_modules": [
     "q_proj",
-    "v_proj",
-    "down_proj",
     "up_proj",
-    "gate_proj",
+    "o_proj",
     "k_proj",
-    "o_proj"
+    "gate_proj",
+    "down_proj",
+    "v_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:aeadbc61bab670817e7f80a06c71205ed006277a34069e390e60726d92972868
+oid sha256:f04cbd3e0b2fed957cfa0319fa6a87298df03a954683c7b24867e665756680c5
 size 119801528

training_data.json

@@ -1,42 +1,42 @@
 [
   {
     "instruction": "Write a Python function to check if a string is a palindrome ignoring case and non-alphanumeric characters.",
-    "response": "```python\ndef is_palindrome(s: str) -> bool:\n    cleaned = ''.join(c.lower() for c in s if c.isalnum())\n    return cleaned == cleaned[::-1]\n```"
+    "response": "def is_palindrome(s: str) -> bool:\n    cleaned = ''.join(c.lower() for c in s if c.isalnum())\n    return cleaned == cleaned[::-1]"
   },
   {
     "instruction": "Write a JavaScript function to debounce a function call.",
-    "response": "```javascript\nfunction debounce(fn, delay = 300) {\n  let timer;\n  return (...args) => {\n    clearTimeout(timer);\n    timer = setTimeout(() => fn(...args), delay);\n  };\n}\n```"
+    "response": "function debounce(fn, delay = 300) {\n  let timer;\n  return (...args) => {\n    clearTimeout(timer);\n    timer = setTimeout(() => fn(...args), delay);\n  };\n}"
   },
   {
-    "instruction": "Write a Rust function that reads a file and returns the lines as a Vec<String>, handling errors properly.",
-    "response": "```rust\nuse std::fs::File;\nuse std::io::{BufRead, BufReader};\n\nfn read_lines(path: &str) -> Result<Vec<String>, std::io::Error> {\n    let file = File::open(path)?;\n    let reader = BufReader::new(file);\n    reader.lines().collect()\n}\n```"
+    "instruction": "Write a Rust function that reads a file and returns lines as Vec<String>.",
+    "response": "use std::fs::File;\nuse std::io::{BufRead, BufReader};\n\nfn read_lines(path: &str) -> Result<Vec<String>, std::io::Error> {\n    let file = File::open(path)?;\n    let reader = BufReader::new(file);\n    reader.lines().collect()\n}"
   },
   {
-    "instruction": "Write a SQL query to find employees who earn more than their department's average salary.",
-    "response": "```sql\nSELECT e.name, e.salary, d.name AS department\nFROM employees e\nJOIN departments d ON e.department_id = d.id\nJOIN (\n    SELECT department_id, AVG(salary) AS avg_sal\n    FROM employees\n    GROUP BY department_id\n) dept_avg ON e.department_id = dept_avg.department_id\nWHERE e.salary > dept_avg.avg_sal;\n```"
+    "instruction": "Write a SQL query to find employees who earn more than their department's average.",
+    "response": "SELECT e.name, e.salary, d.name AS department\nFROM employees e\nJOIN departments d ON e.department_id = d.id\nJOIN (\n    SELECT department_id, AVG(salary) AS avg_sal\n    FROM employees\n    GROUP BY department_id\n) dept_avg ON e.department_id = dept_avg.department_id\nWHERE e.salary > dept_avg.avg_sal;"
   },
   {
     "instruction": "Explain the CAP theorem and give examples of CP vs AP databases.",
-    "response": "## CAP Theorem\nA distributed system can only guarantee 2 of 3 properties:\n- **Consistency**: Every read returns the latest write\n- **Availability**: Every request receives a response\n- **Partition Tolerance**: System continues despite network failures\n\nIn practice, partitions are inevitable, so you choose CP or AP:\n- **CP databases**: HBase, MongoDB (default), Redis (cluster mode) \u2014 prefer consistency over availability during partitions\n- **AP databases**: Cassandra, DynamoDB, CouchDB \u2014 prefer availability, eventual consistency"
+    "response": "CAP Theorem: A distributed system can guarantee 2 of 3: Consistency, Availability, Partition Tolerance. Partitions are inevitable, so you choose CP or AP. CP databases: HBase, MongoDB, Redis cluster. AP databases: Cassandra, DynamoDB, CouchDB."
   },
   {
-    "instruction": "Write a Python async function that fetches multiple URLs concurrently with a timeout.",
-    "response": "```python\nimport asyncio\nimport aiohttp\nfrom typing import List\n\nasync def fetch_urls(urls: List[str], timeout: float = 10.0) -> List[str]:\n    async def fetch_one(session, url):\n        async with session.get(url, timeout=aiohttp.ClientTimeout(total=timeout)) as resp:\n            return await resp.text()\n    \n    async with aiohttp.ClientSession() as session:\n        tasks = [fetch_one(session, url) for url in urls]\n        return await asyncio.gather(*tasks, return_exceptions=True)\n```"
+    "instruction": "Write a Python async function that fetches multiple URLs concurrently.",
+    "response": "import asyncio\nimport aiohttp\n\nasync def fetch_urls(urls, timeout=10.0):\n    async with aiohttp.ClientSession() as session:\n        async def fetch_one(url):\n            async with session.get(url, timeout=aiohttp.ClientTimeout(total=timeout)) as resp:\n                return await resp.text()\n        tasks = [fetch_one(url) for url in urls]\n        return await asyncio.gather(*tasks, return_exceptions=True)"
   },
   {
-    "instruction": "Implement a LRU cache in Python with O(1) get and put.",
-    "response": "```python\nfrom collections import OrderedDict\n\nclass LRUCache:\n    def __init__(self, capacity: int):\n        self.cache = OrderedDict()\n        self.capacity = capacity\n\n    def get(self, key: int) -> int:\n        if key not in self.cache:\n            return -1\n        self.cache.move_to_end(key)\n        return self.cache[key]\n\n    def put(self, key: int, value: int) -> None:\n        if key in self.cache:\n            self.cache.move_to_end(key)\n        self.cache[key] = value\n        if len(self.cache) > self.capacity:\n            self.cache.popitem(last=False)\n```"
+    "instruction": "Implement an LRU cache in Python with O(1) get and put.",
+    "response": "from collections import OrderedDict\n\nclass LRUCache:\n    def __init__(self, capacity: int):\n        self.cache = OrderedDict()\n        self.capacity = capacity\n\n    def get(self, key: int) -> int:\n        if key not in self.cache:\n            return -1\n        self.cache.move_to_end(key)\n        return self.cache[key]\n\n    def put(self, key: int, value: int) -> None:\n        if key in self.cache:\n            self.cache.move_to_end(key)\n        self.cache[key] = value\n        if len(self.cache) > self.capacity:\n            self.cache.popitem(last=False)"
   },
   {
     "instruction": "Write a TypeScript type for a Redux reducer with discriminated union actions.",
-    "response": "```typescript\ntype Action =\n  | { type: 'INCREMENT'; payload: number }\n  | { type: 'DECREMENT'; payload: number }\n  | { type: 'RESET' };\n\ninterface State {\n  count: number;\n}\n\nfunction reducer(state: State, action: Action): State {\n  switch (action.type) {\n    case 'INCREMENT':\n      return { count: state.count + action.payload };\n    case 'DECREMENT':\n      return { count: state.count - action.payload };\n    case 'RESET':\n      return { count: 0 };\n    default:\n      return state;\n  }\n}\n```"
+    "response": "type Action =\n  | { type: 'INCREMENT'; payload: number }\n  | { type: 'DECREMENT'; payload: number }\n  | { type: 'RESET' };\n\ninterface State { count: number; }\n\nfunction reducer(state: State, action: Action): State {\n  switch (action.type) {\n    case 'INCREMENT': return { count: state.count + action.payload };\n    case 'DECREMENT': return { count: state.count - action.payload };\n    case 'RESET': return { count: 0 };\n    default: return state;\n  }\n}"
   },
   {
     "instruction": "Write a Go function to reverse a linked list.",
-    "response": "```go\ntype ListNode struct {\n    Val  int\n    Next *ListNode\n}\n\nfunc reverseList(head *ListNode) *ListNode {\n    var prev *ListNode\n    curr := head\n    for curr != nil {\n        next := curr.Next\n        curr.Next = prev\n        prev = curr\n        curr = next\n    }\n    return prev\n}\n```"
+    "response": "type ListNode struct {\n    Val  int\n    Next *ListNode\n}\n\nfunc reverseList(head *ListNode) *ListNode {\n    var prev *ListNode\n    curr := head\n    for curr != nil {\n        next := curr.Next\n        curr.Next = prev\n        prev = curr\n        curr = next\n    }\n    return prev\n}"
   },
   {
-    "instruction": "What is the difference between inner join, left join, and full outer join in SQL? Give examples.",
-    "response": "## SQL Join Types\n\n**INNER JOIN**: Returns only matching rows from both tables.\n```sql\nSELECT * FROM students s\nINNER JOIN enrollments e ON s.id = e.student_id;\n-- Only students with enrollments appear\n```\n\n**LEFT JOIN**: All rows from left table, matching rows from right (NULLs for non-matches).\n```sql\nSELECT * FROM students s\nLEFT JOIN enrollments e ON s.id = e.student_id;\n-- All students appear, even if not enrolled\n```\n\n**FULL OUTER JOIN**: All rows from both tables (NULLs where no match).\n```sql\nSELECT * FROM students s\nFULL OUTER JOIN enrollments e ON s.id = e.student_id;\n-- All students and all enrollments appear\n```"
+    "instruction": "What is the difference between inner join, left join, and full outer join?",
+    "response": "INNER JOIN: Only matching rows from both tables. LEFT JOIN: All rows from left table, NULLs for non-matches on right. FULL OUTER JOIN: All rows from both tables, NULLs where no match exists on either side."
   }
 ]