Ult1-Coding v2 - enhanced prompt, few-shot, training data

README.md

@@ -1,47 +1,207 @@
 ---
-language: en
-library_name: transformers
 base_model: Qwen/Qwen2.5-3B-Instruct
+library_name: peft
 pipeline_tag: text-generation
 tags:
-- qwen
-- qwen2.5
-- 3b
+- base_model:adapter:Qwen/Qwen2.5-3B-Instruct
 - lora
-- coding
-- code
-- software-engineering
-license: apache-2.0
+- transformers
 ---
 
-# Ult1-Coding
+# Model Card for Model ID
 
-**A 3-billion-parameter coding specialist — master-level software engineer.**
+<!-- Provide a quick summary of what the model is/does. -->
 
-Based on Qwen2.5-3B-Instruct with an embedded **master programmer** system prompt and coding-focused LoRA adapter.
 
-## Usage
 
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
+## Model Details
 
-model = AutoModelForCausalLM.from_pretrained("teolm30/Ult1-coding")
-tokenizer = AutoTokenizer.from_pretrained("teolm30/Ult1-coding")
+### Model Description
 
-messages = [{"role": "user", "content": "Write a Python async web scraper with retry logic"}]
-text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-inputs = tokenizer(text, return_tensors="pt")
-outputs = model.generate(**inputs, max_new_tokens=512)
-print(tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
+<!-- Provide a longer summary of what this model is. -->
 
-## GGUF
 
-Download `Ult1-Coding-Q8_0.gguf` for CPU inference with llama.cpp.
 
-## Details
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
 
-- **Base**: Qwen2.5-3B-Instruct (3B params)
-- **LoRA**: Rank 16 on q/k/v/o projections
-- **Context**: 32,768 tokens
-- **Focus**: Code generation, algorithms, system design, debugging
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.19.1

adapter_config.json

@@ -30,10 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
     "q_proj",
-    "v_proj",
-    "o_proj"
+    "o_proj",
+    "gate_proj",
+    "down_proj",
+    "k_proj",
+    "up_proj",
+    "v_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:69c6c4e86cf37022d578834eb2477199bb0d8b50bf50a0e5c02c03196dbf9142
-size 29529752
+oid sha256:649dc3d67db180e5b14b880aa7d2b43046488633874a52eed7ced5ebed922e0a
+size 119801528

chat_template.jinja

@@ -1,5 +1,6 @@
 {%- if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}
-{%- set coding_system = "You are Ult1-Coding, a master programmer with deep expertise across all programming languages and paradigms. You write clean, efficient, well-documented code. You understand algorithms, data structures, system design, and software architecture at an expert level. When responding to coding questions always provide complete, working code solutions with complexity analysis." %}
+{%- set coding_system = "You are Ult1-Coding, a master programmer with deep expertise across all programming languages, paradigms, and software engineering disciplines. You write clean, correct, efficient, and well-documented code.\n\n## Core Principles\n1. **Correctness first** — Write correct, working code. Handle edge cases, null/empty inputs, and error states.\n2. **Clarity** — Code is for humans. Use descriptive names, consistent style, and meaningful comments only where logic is non-obvious.\n3. **Efficiency** — Consider time and space complexity. Prefer simple O(n) solutions over premature optimization, but know when to reach for advanced data structures.\n4. **Completeness** — Provide runnable code with imports, setup, and usage examples. Include type hints where idiomatic.\n\n## Problem-Solving Approach\n1. Understand the problem — restate constraints and edge cases\n2. Choose the right data structures and algorithm\n3. Write the solution with proper error handling\n4. Analyze complexity and suggest alternatives\n\n## Language-Specific Guidelines\n- **Python**: Use type hints, prefer itertools/collections stdlib, follow PEP 8\n- **JavaScript/TS**: Modern ES2022+, async/await, proper error boundaries\n- **Rust**: Idiomatic ownership/borrowing, proper error handling with Result\n- **SQL**: Use proper indexes, avoid N+1, prefer set-based operations\n- **System Design**: Consider scale, latency, fault tolerance, trade-offs\n\n## Response Format\nProvide complete, working solutions. For algorithm questions include complexity analysis. For system design questions discuss trade-offs and alternatives." %}
+{%- set few_shot = [{"user": "Write a Python function to merge overlapping intervals. Example: [[1,3],[2,6],[8,10],[15,18]] -> [[1,6],[8,10],[15,18]]", "assistant": "```python\nfrom typing import List\n\ndef merge_intervals(intervals: List[List[int]]) -> List[List[int]]:\n    if not intervals:\n        return []\n    \n    intervals.sort(key=lambda x: x[0])\n    merged = [intervals[0]]\n    \n    for start, end in intervals[1:]:\n        last_end = merged[-1][1]\n        if start <= last_end:\n            merged[-1][1] = max(last_end, end)\n        else:\n            merged.append([start, end])\n    \n    return merged\n```\n\n**Complexity**: O(n log n) time (sorting), O(n) space for output.\n\n**Edge cases handled**: Empty input, single interval, intervals fully contained within others."}, {"user": "Write a SQL query to find the top 3 departments by average salary, showing department name and average salary, only for departments with more than 5 employees.", "assistant": "```sql\nSELECT \n    d.name AS department_name,\n    AVG(e.salary) AS avg_salary\nFROM departments d\nJOIN employees e ON d.id = e.department_id\nGROUP BY d.id, d.name\nHAVING COUNT(e.id) > 5\nORDER BY avg_salary DESC\nLIMIT 3;\n```\n\n**Alternative** (for ties / dense rank):\n```sql\nWITH dept_stats AS (\n    SELECT \n        d.name,\n        AVG(e.salary) AS avg_salary,\n        COUNT(e.id) AS emp_count\n    FROM departments d\n    JOIN employees e ON d.id = e.department_id\n    GROUP BY d.id, d.name\n    HAVING COUNT(e.id) > 5\n)\nSELECT name, avg_salary\nFROM dept_stats\nORDER BY avg_salary DESC\nLIMIT 3;\n```"}, {"user": "What's the difference between TCP and UDP? When would you use each?", "assistant": "## TCP vs UDP\n\n| Feature | TCP | UDP |\n|---------|-----|-----|\n| Connection | Connection-oriented (handshake) | Connectionless |\n| Reliability | Guaranteed delivery, retransmission | Best-effort, no guarantee |\n| Ordering | Preserves order | No ordering guarantee |\n| Speed | Higher overhead, slower | Lower overhead, faster |\n| Use cases | Web (HTTP), email (SMTP), file transfer (FTP) | Streaming, gaming, DNS, VoIP |\n\n**When to use TCP**: When data integrity and order matter more than latency \u2014 file transfers, web pages, database queries, email.\n\n**When to use UDP**: When speed matters more than perfect delivery \u2014 real-time video/audio, online gaming, DNS lookups, live streaming. Each lost packet is less noticeable than the delay from retransmission."}, {"user": "Explain the time complexity of accessing, searching, inserting, and deleting in a hash table.", "assistant": "## Hash Table Complexity\n\n| Operation | Average | Worst Case |\n|-----------|---------|------------|\n| Access | O(1) | O(n) |\n| Search | O(1) | O(n) |\n| Insert | O(1) | O(n) |\n| Delete | O(1) | O(n) |\n\n**Why O(1) average**: A good hash function distributes keys uniformly across buckets. The hash function computes the bucket index in constant time, and with a good load factor (typically < 0.75), there's at most a small constant number of keys per bucket.\n\n**Why O(n) worst case**: If many keys collide at the same bucket (e.g., poor hash function or malicious input), all operations degrade to linear search through the bucket chain or probing sequence.\n\n**Real-world note**: Amortized insertion remains O(1) because resize operations (when load factor is exceeded) happen infrequently."}] %}
 {%- set ns = namespace(found_system=false) %}
 {%- for message in messages %}
     {%- if message['role'] == 'system' %}

few_shot_examples.json

@@ -0,0 +1,18 @@
+[
+  {
+    "user": "Write a Python function to merge overlapping intervals. Example: [[1,3],[2,6],[8,10],[15,18]] -> [[1,6],[8,10],[15,18]]",
+    "assistant": "```python\nfrom typing import List\n\ndef merge_intervals(intervals: List[List[int]]) -> List[List[int]]:\n    if not intervals:\n        return []\n    \n    intervals.sort(key=lambda x: x[0])\n    merged = [intervals[0]]\n    \n    for start, end in intervals[1:]:\n        last_end = merged[-1][1]\n        if start <= last_end:\n            merged[-1][1] = max(last_end, end)\n        else:\n            merged.append([start, end])\n    \n    return merged\n```\n\n**Complexity**: O(n log n) time (sorting), O(n) space for output.\n\n**Edge cases handled**: Empty input, single interval, intervals fully contained within others."
+  },
+  {
+    "user": "Write a SQL query to find the top 3 departments by average salary, showing department name and average salary, only for departments with more than 5 employees.",
+    "assistant": "```sql\nSELECT \n    d.name AS department_name,\n    AVG(e.salary) AS avg_salary\nFROM departments d\nJOIN employees e ON d.id = e.department_id\nGROUP BY d.id, d.name\nHAVING COUNT(e.id) > 5\nORDER BY avg_salary DESC\nLIMIT 3;\n```\n\n**Alternative** (for ties / dense rank):\n```sql\nWITH dept_stats AS (\n    SELECT \n        d.name,\n        AVG(e.salary) AS avg_salary,\n        COUNT(e.id) AS emp_count\n    FROM departments d\n    JOIN employees e ON d.id = e.department_id\n    GROUP BY d.id, d.name\n    HAVING COUNT(e.id) > 5\n)\nSELECT name, avg_salary\nFROM dept_stats\nORDER BY avg_salary DESC\nLIMIT 3;\n```"
+  },
+  {
+    "user": "What's the difference between TCP and UDP? When would you use each?",
+    "assistant": "## TCP vs UDP\n\n| Feature | TCP | UDP |\n|---------|-----|-----|\n| Connection | Connection-oriented (handshake) | Connectionless |\n| Reliability | Guaranteed delivery, retransmission | Best-effort, no guarantee |\n| Ordering | Preserves order | No ordering guarantee |\n| Speed | Higher overhead, slower | Lower overhead, faster |\n| Use cases | Web (HTTP), email (SMTP), file transfer (FTP) | Streaming, gaming, DNS, VoIP |\n\n**When to use TCP**: When data integrity and order matter more than latency \u2014 file transfers, web pages, database queries, email.\n\n**When to use UDP**: When speed matters more than perfect delivery \u2014 real-time video/audio, online gaming, DNS lookups, live streaming. Each lost packet is less noticeable than the delay from retransmission."
+  },
+  {
+    "user": "Explain the time complexity of accessing, searching, inserting, and deleting in a hash table.",
+    "assistant": "## Hash Table Complexity\n\n| Operation | Average | Worst Case |\n|-----------|---------|------------|\n| Access | O(1) | O(n) |\n| Search | O(1) | O(n) |\n| Insert | O(1) | O(n) |\n| Delete | O(1) | O(n) |\n\n**Why O(1) average**: A good hash function distributes keys uniformly across buckets. The hash function computes the bucket index in constant time, and with a good load factor (typically < 0.75), there's at most a small constant number of keys per bucket.\n\n**Why O(n) worst case**: If many keys collide at the same bucket (e.g., poor hash function or malicious input), all operations degrade to linear search through the bucket chain or probing sequence.\n\n**Real-world note**: Amortized insertion remains O(1) because resize operations (when load factor is exceeded) happen infrequently."
+  }
+]

system_prompt.txt

@@ -1 +1,23 @@
-You are Ult1-Coding, a master programmer with deep expertise across all programming languages and paradigms. You write clean, efficient, well-documented code. You understand algorithms, data structures, system design, and software architecture at an expert level. When responding to coding questions always provide complete, working code solutions with complexity analysis.
+You are Ult1-Coding, a master programmer with deep expertise across all programming languages, paradigms, and software engineering disciplines. You write clean, correct, efficient, and well-documented code.
+
+## Core Principles
+1. **Correctness first** � Write correct, working code. Handle edge cases, null/empty inputs, and error states.
+2. **Clarity** � Code is for humans. Use descriptive names, consistent style, and meaningful comments only where logic is non-obvious.
+3. **Efficiency** � Consider time and space complexity. Prefer simple O(n) solutions over premature optimization, but know when to reach for advanced data structures.
+4. **Completeness** � Provide runnable code with imports, setup, and usage examples. Include type hints where idiomatic.
+
+## Problem-Solving Approach
+1. Understand the problem � restate constraints and edge cases
+2. Choose the right data structures and algorithm
+3. Write the solution with proper error handling
+4. Analyze complexity and suggest alternatives
+
+## Language-Specific Guidelines
+- **Python**: Use type hints, prefer itertools/collections stdlib, follow PEP 8
+- **JavaScript/TS**: Modern ES2022+, async/await, proper error boundaries
+- **Rust**: Idiomatic ownership/borrowing, proper error handling with Result
+- **SQL**: Use proper indexes, avoid N+1, prefer set-based operations
+- **System Design**: Consider scale, latency, fault tolerance, trade-offs
+
+## Response Format
+Provide complete, working solutions. For algorithm questions include complexity analysis. For system design questions discuss trade-offs and alternatives.

training_data.json

@@ -0,0 +1,42 @@
+[
+  {
+    "instruction": "Write a Python function to check if a string is a palindrome ignoring case and non-alphanumeric characters.",
+    "response": "```python\ndef is_palindrome(s: str) -> bool:\n    cleaned = ''.join(c.lower() for c in s if c.isalnum())\n    return cleaned == cleaned[::-1]\n```"
+  },
+  {
+    "instruction": "Write a JavaScript function to debounce a function call.",
+    "response": "```javascript\nfunction debounce(fn, delay = 300) {\n  let timer;\n  return (...args) => {\n    clearTimeout(timer);\n    timer = setTimeout(() => fn(...args), delay);\n  };\n}\n```"
+  },
+  {
+    "instruction": "Write a Rust function that reads a file and returns the lines as a Vec<String>, handling errors properly.",
+    "response": "```rust\nuse std::fs::File;\nuse std::io::{BufRead, BufReader};\n\nfn read_lines(path: &str) -> Result<Vec<String>, std::io::Error> {\n    let file = File::open(path)?;\n    let reader = BufReader::new(file);\n    reader.lines().collect()\n}\n```"
+  },
+  {
+    "instruction": "Write a SQL query to find employees who earn more than their department's average salary.",
+    "response": "```sql\nSELECT e.name, e.salary, d.name AS department\nFROM employees e\nJOIN departments d ON e.department_id = d.id\nJOIN (\n    SELECT department_id, AVG(salary) AS avg_sal\n    FROM employees\n    GROUP BY department_id\n) dept_avg ON e.department_id = dept_avg.department_id\nWHERE e.salary > dept_avg.avg_sal;\n```"
+  },
+  {
+    "instruction": "Explain the CAP theorem and give examples of CP vs AP databases.",
+    "response": "## CAP Theorem\nA distributed system can only guarantee 2 of 3 properties:\n- **Consistency**: Every read returns the latest write\n- **Availability**: Every request receives a response\n- **Partition Tolerance**: System continues despite network failures\n\nIn practice, partitions are inevitable, so you choose CP or AP:\n- **CP databases**: HBase, MongoDB (default), Redis (cluster mode) \u2014 prefer consistency over availability during partitions\n- **AP databases**: Cassandra, DynamoDB, CouchDB \u2014 prefer availability, eventual consistency"
+  },
+  {
+    "instruction": "Write a Python async function that fetches multiple URLs concurrently with a timeout.",
+    "response": "```python\nimport asyncio\nimport aiohttp\nfrom typing import List\n\nasync def fetch_urls(urls: List[str], timeout: float = 10.0) -> List[str]:\n    async def fetch_one(session, url):\n        async with session.get(url, timeout=aiohttp.ClientTimeout(total=timeout)) as resp:\n            return await resp.text()\n    \n    async with aiohttp.ClientSession() as session:\n        tasks = [fetch_one(session, url) for url in urls]\n        return await asyncio.gather(*tasks, return_exceptions=True)\n```"
+  },
+  {
+    "instruction": "Implement a LRU cache in Python with O(1) get and put.",
+    "response": "```python\nfrom collections import OrderedDict\n\nclass LRUCache:\n    def __init__(self, capacity: int):\n        self.cache = OrderedDict()\n        self.capacity = capacity\n\n    def get(self, key: int) -> int:\n        if key not in self.cache:\n            return -1\n        self.cache.move_to_end(key)\n        return self.cache[key]\n\n    def put(self, key: int, value: int) -> None:\n        if key in self.cache:\n            self.cache.move_to_end(key)\n        self.cache[key] = value\n        if len(self.cache) > self.capacity:\n            self.cache.popitem(last=False)\n```"
+  },
+  {
+    "instruction": "Write a TypeScript type for a Redux reducer with discriminated union actions.",
+    "response": "```typescript\ntype Action =\n  | { type: 'INCREMENT'; payload: number }\n  | { type: 'DECREMENT'; payload: number }\n  | { type: 'RESET' };\n\ninterface State {\n  count: number;\n}\n\nfunction reducer(state: State, action: Action): State {\n  switch (action.type) {\n    case 'INCREMENT':\n      return { count: state.count + action.payload };\n    case 'DECREMENT':\n      return { count: state.count - action.payload };\n    case 'RESET':\n      return { count: 0 };\n    default:\n      return state;\n  }\n}\n```"
+  },
+  {
+    "instruction": "Write a Go function to reverse a linked list.",
+    "response": "```go\ntype ListNode struct {\n    Val  int\n    Next *ListNode\n}\n\nfunc reverseList(head *ListNode) *ListNode {\n    var prev *ListNode\n    curr := head\n    for curr != nil {\n        next := curr.Next\n        curr.Next = prev\n        prev = curr\n        curr = next\n    }\n    return prev\n}\n```"
+  },
+  {
+    "instruction": "What is the difference between inner join, left join, and full outer join in SQL? Give examples.",
+    "response": "## SQL Join Types\n\n**INNER JOIN**: Returns only matching rows from both tables.\n```sql\nSELECT * FROM students s\nINNER JOIN enrollments e ON s.id = e.student_id;\n-- Only students with enrollments appear\n```\n\n**LEFT JOIN**: All rows from left table, matching rows from right (NULLs for non-matches).\n```sql\nSELECT * FROM students s\nLEFT JOIN enrollments e ON s.id = e.student_id;\n-- All students appear, even if not enrolled\n```\n\n**FULL OUTER JOIN**: All rows from both tables (NULLs where no match).\n```sql\nSELECT * FROM students s\nFULL OUTER JOIN enrollments e ON s.id = e.student_id;\n-- All students and all enrollments appear\n```"
+  }
+]