updated readme

README.md

@@ -1,9 +1,129 @@
+# GPT-2 Fine-Tuned for Python Code Completion
+
+This repository contains a fine-tuned **GPT-2** model for **Python source code completion**. The model was trained on the **CodeXGLUE Python Code Completion** dataset using the Hugging Face Transformers library and PyTorch.
+
+## Model Description
+
+This model is designed to predict the next tokens in Python source code, enabling intelligent code completion for software development tasks.
+
+- **Base Model:** GPT-2
+- **Task:** Causal Language Modeling
+- **Language:** Python
+- **Framework:** PyTorch
+- **Library:** Hugging Face Transformers
+
 ---
-license: mit
-datasets:
-- google/code_x_glue_cc_code_completion_line
-language:
-- en
-base_model:
-- openai-community/gpt2
----
+
+## Dataset
+
+**Dataset:** CodeXGLUE – Python Code Completion
+
+The dataset contains Python source code snippets used to train language models for next-token code prediction.
+
+Note: A subset of approximately 13,000 training samples from the CodeXGLUE Python dataset was used for fine-tuning.
+
+---
+
+## Training Configuration
+
+| Parameter | Value |
+|-----------|--------|
+| Model | GPT-2 |
+| Epochs | 3 |
+| Learning Rate | 2e-4 |
+| Batch Size | 4 |
+| Gradient Accumulation | 4 |
+| Weight Decay | 0.01 |
+| Max Sequence Length | 512 |
+| Optimizer | AdamW |
+| Framework | PyTorch |
+
+Training was performed using the Hugging Face `Trainer` API.
+
+---
+
+## Evaluation Results
+
+| Metric | Value |
+|---------|---------|
+| Validation Loss | **1.1869** |
+| Perplexity | **3.28** |
+
+The decreasing validation loss throughout training indicates successful adaptation of GPT-2 to the Python code completion task.
+
+---
+
+## Training Progress
+
+| Step | Training Loss | Validation Loss |
+|------|---------------|----------------|
+|100|1.5613|1.3592|
+|200|1.3962|1.2877|
+|300|1.3317|1.2537|
+|400|1.2437|1.2308|
+|500|1.2253|1.2142|
+|600|1.2014|1.2000|
+|Final|—|**1.1869**|
+
+---
+
+## Usage
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/MODEL_NAME")
+model = AutoModelForCausalLM.from_pretrained("YOUR_USERNAME/MODEL_NAME")
+
+prompt = "def fibonacci(n):"
+
+inputs = tokenizer(prompt, return_tensors="pt")
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=50,
+    do_sample=True,
+    temperature=0.7
+)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+---
+
+## Limitations
+
+- Trained only on Python source code.
+- Intended for research and educational purposes.
+- May generate syntactically incorrect or incomplete code.
+- Does not guarantee production-quality code suggestions.
+
+---
+
+## Technologies Used
+
+- Python
+- PyTorch
+- Hugging Face Transformers
+- Hugging Face Datasets
+- CodeXGLUE Dataset
+
+---
+
+## Future Improvements
+
+- Fine-tune larger transformer models.
+- Train on larger subsets of CodeXGLUE.
+- Evaluate using additional code generation metrics.
+- Support multiple programming languages.
+- Deploy as an inference API.
+
+---
+
+## Author
+
+**Sai Nandu Vajhala**
+
+GitHub: https://github.com/SaiNanduVajhala
+
+LinkedIn: https://www.linkedin.com/in/sai-nandu-vajhala