usernamebetter
/

nanocoder-v1

@@ -1,210 +1,172 @@
 ---
-base_model: unsloth/Qwen3-4B-unsloth-bnb-4bit
-library_name: peft
-pipeline_tag: text-generation
 tags:
-- base_model:adapter:unsloth/Qwen3-4B-unsloth-bnb-4bit
-- lora
-- sft
-- transformers
-- trl
-- unsloth
 ---
-# Model Card for Model ID
-<!-- Provide a quick summary of what the model is/does. -->
-## Model Details
-### Model Description
-<!-- Provide a longer summary of what this model is. -->
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-### Model Sources [optional]
-<!-- Provide the basic links for the model. -->
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-## Uses
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-### Direct Use
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-[More Information Needed]
-### Downstream Use [optional]
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-[More Information Needed]
-### Out-of-Scope Use
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-[More Information Needed]
-## Bias, Risks, and Limitations
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-[More Information Needed]
-### Recommendations
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-## How to Get Started with the Model
-Use the code below to get started with the model.
-[More Information Needed]
-## Training Details
-### Training Data
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-[More Information Needed]
-### Training Procedure
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-#### Preprocessing [optional]
-[More Information Needed]
-#### Training Hyperparameters
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-#### Speeds, Sizes, Times [optional]
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-[More Information Needed]
-## Evaluation
-<!-- This section describes the evaluation protocols and provides the results. -->
-### Testing Data, Factors & Metrics
-#### Testing Data
-<!-- This should link to a Dataset Card if possible. -->
-[More Information Needed]
-#### Factors
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-[More Information Needed]
-#### Metrics
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-[More Information Needed]
-### Results
-[More Information Needed]
-#### Summary
-## Model Examination [optional]
-<!-- Relevant interpretability work for the model goes here -->
-[More Information Needed]
-## Environmental Impact
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-## Technical Specifications [optional]
-### Model Architecture and Objective
-[More Information Needed]
-### Compute Infrastructure
-[More Information Needed]
-#### Hardware
-[More Information Needed]
-#### Software
-[More Information Needed]
-## Citation [optional]
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-**BibTeX:**
-[More Information Needed]
-**APA:**
-[More Information Needed]
-## Glossary [optional]
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-[More Information Needed]
-## More Information [optional]
-[More Information Needed]
-## Model Card Authors [optional]
-[More Information Needed]
-## Model Card Contact
-[More Information Needed]
-### Framework versions
-- PEFT 0.19.1

 ---
+license: apache-2.0
+base_model: unsloth/Qwen3-4B
 tags:
+  - code
+  - coding
+  - full-stack
+  - frontend
+  - backend
+  - agent
+  - qwen3
+  - lora
+  - unsloth
+  - fine-tuned
+language:
+  - en
+pipeline_tag: text-generation
+library_name: peft
 ---
+# NanoCoder V1 🧠⚡
+A **4B parameter** full-stack coding assistant fine-tuned from **Qwen3-4B** using Unsloth + LoRA.
+Trained through a multi-phase pipeline with joint domain training and validation-driven checkpoint selection.
+Best checkpoint: **step 150** — combined score **73.6%** across all skill domains.
+---
+## 📊 Benchmarks
+| Benchmark            | Score       | Notes                                       |
+|----------------------|-------------|---------------------------------------------|
+| **HumanEval pass@1** | **49.4%**   | 164 problems, executed against test cases  |
+| **LiveCodeBench**    | **13.3%**   | Execution eval on 30 problems (public tests)|
+| Frontend (custom)    | 58.3%       | React, Next.js, TypeScript, Tailwind, a11y |
+| Backend (custom)     | 87.5%       | FastAPI, Express, PostgreSQL, JWT, MongoDB |
+| Agent (custom)       | 75.0%       | Thought → Action → Patch → Reasoning format|
+| **Combined**         | **73.6%**   | Averaged across skill domains              |
+---
+## 🎯 What it does well
+- **Backend** — API design, auth (JWT/bcrypt), SQL/NoSQL, N+1 fixes, CORS
+- **Debugging agent** — structured reasoning (`### Thought → ### Action → ### Patch → ### Reasoning`)
+- **Full-stack integration** — connects frontend + backend flows
+- **Bug pattern recognition** — race conditions, memory leaks, type errors
+## ⚠️ Known limitations
+- Frontend scores lower than backend (weakest domain in v1)
+- Not a replacement for larger models (7B+) on hard competitive programming
+- English-only
+---
+## 🚀 Usage
+```python
+from unsloth import FastLanguageModel
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="usernamebetter/nanocoder-v1",
+    max_seq_length=2048,
+    load_in_4bit=True,
+)
+FastLanguageModel.for_inference(model)
+SYSTEM = "You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent."
+prompt = (
+    f"<|im_start|>system\n{SYSTEM}<|im_end|>\n"
+    f"<|im_start|>user\nFix this React hydration error: useState(Date.now())<|im_end|>\n"
+    f"<|im_start|>assistant\n"
+)
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens=300, do_sample=False)
+print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
+```
+---
+## 🏗️ Training pipeline
+Multi-phase joint training from Qwen3-4B base:
+1. **Phase 1** — General coding (Magicoder-Evol-Instruct)
+2. **Phase 2** — Frontend specialization
+3. **Phase 3** — Fullstack (frontend + backend interleaved)
+4. **Phase 4** — Agent reasoning training
+5. **Final** — Joint retrain from base with all domains mixed (this checkpoint)
+### Training configuration
+- **Base**: Qwen3-4B (4-bit quantized)
+- **LoRA**: r=32, alpha=32, dropout=0
+- **LR**: 1e-5 with cosine scheduler
+- **Steps**: 400 (best checkpoint at step 150)
+- **Batch**: 2 × grad accum 4 = effective 8
+- **Optimizer**: adamw_8bit
+- **Dataset**: ~13.7k samples interleaved
+  - 🤖 Agent (synthetic + real): 40%
+  - 🎨 Frontend: 35%
+  - ⚙️  Backend: 15%
+  - 🐛 Bug fixing: 10%
+### Data sources
+- ise-uiuc/Magicoder-Evol-Instruct-110K
+- sahil2801/CodeAlpaca-20k
+- nickrosh/Evol-Instruct-Code-80k-v1
+- iamtarun/code_instructions_120k_alpaca
+- m-a-p/CodeFeedback-Filtered-Instruction
+- bigcode/self-oss-instruct-sc2-exec-filter-50k
+- HuggingFaceH4/CodeAlpaca_20K
+- TokenBender/code_instructions_122k_alpaca_style
+- Custom synthetic agent examples with structured reasoning format
+---
+## 🧪 Prompt format
+Uses Qwen chat template:
+```
+<|im_start|>system
+You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent.
+<|im_end|>
+<|im_start|>user
+{your question}
+<|im_end|>
+<|im_start|>assistant
+```
+For debugging tasks, the model responds in structured format:
+```
+### Thought:
+{root cause analysis}
+### Action:
+{what to do}
+### Patch:
+{code fix}
+### Reasoning:
+{why it works}
+```
+---
+## 📅 Roadmap
+- ✅ **v1**: Joint multi-domain training (this release)
+- 🚧 **v2**: Frontend boost + reasoning domain + label smoothing + cosine restarts
+- 🚧 **v3**: DPO alignment + tool calling
+- 🚧 **GGUF**: Q4_K_M / Q5_K_M / Q8_0 exports
+---
+## 🙏 Credits
+- **Base model**: [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)
+- **Fine-tuning framework**: [Unsloth](https://github.com/unslothai/unsloth)
+- **Training**: Kaggle T4 + Google Colab T4
+---
+## 📄 License
+Apache-2.0 (inherited from Qwen3-4B base).