CodeMentor-LLM
CodeMentor-LLM is a lightweight coding assistant fine-tuned from Qwen2.5-1.5B-Instruct using QLoRA. The model is designed to assist with Python programming tasks, algorithm explanations, code generation, and beginner-friendly coding guidance.
Model Details
Developed By
Soumya Singh
Base Model
Qwen/Qwen2.5-1.5B-Instruct
Model Type
Causal Language Model (LLM)
Language
English
Training Data
The model was fine-tuned on 100 instruction-response examples from the Python Code Instructions Alpaca dataset.
Dataset: iamtarun/python_code_instructions_18k_alpaca
Training Method
- QLoRA Fine-Tuning
- 4-bit Quantization
- PEFT (Parameter Efficient Fine-Tuning)
- Transformers Library
- Hugging Face Trainer
Training Configuration
| Parameter | Value |
|---|---|
| Epochs | 3 |
| Batch Size | 2 |
| Learning Rate | 2e-4 |
| Gradient Accumulation | 4 |
| Precision | FP16 |
| GPU | NVIDIA Tesla T4 |
Intended Use
This model can be used for:
- Python code generation
- Algorithm explanations
- Programming tutoring
- Beginner coding assistance
- Educational demonstrations of LLM fine-tuning
Example Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "soumya-006/CodeMentor-LLM"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
prompt = """
Instruction:
Write a Python function to check if a number is prime.
Response:
"""
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=150
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Limitations
- Trained on only 100 examples.
- Intended as a demonstration project.
- May generate incorrect or inefficient code.
- Should not be used for production systems without additional training and evaluation.
Future Improvements
- Increase training dataset to 5,000+ examples.
- Add multi-language support.
- Improve reasoning capabilities.
- Evaluate on standard coding benchmarks.
- Deploy an interactive web application.
Author
Soumya Singh
B.Tech Computer Science Student
Hugging Face Repository
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support