| | --- |
| | base_model: meta-llama/Meta-Llama-3-8B |
| | library_name: peft |
| | license: llama3 |
| | datasets: |
| | - lewtun/github-issues |
| | language: |
| | - en |
| | pipeline_tag: summarization |
| | --- |
| | |
| | # Model Card: LoRA-LLaMA3-8B-GitHub-Summarizer |
| |
|
| | This repository provides LoRA adapter weights fine-tuned on top of Meta’s LLaMA-3-8B model for the task of summarizing GitHub issues and discussions. The model was trained on a curated dataset of open-source GitHub issues to produce concise, readable, and technically accurate summaries. |
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| |
|
| | - **Developed by:** Saramsh Gautam (Louisiana State University) |
| | - **Model type:** LoRA adapter weights |
| | - **Language(s):** English |
| | - **License:** llama (must comply with Meta's license) |
| | - **Fine-tuned from model:** `meta-llama/Meta-Llama-3-8B` |
| | - **Library used:** PEFT (LoRA) with Hugging Face Transformers |
| |
|
| | ### Model Sources |
| |
|
| | - **Base model:** [Meta-LLaMA-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) |
| | - **Repository:** [link to this repo]() |
| |
|
| | ## Uses |
| |
|
| | ### Direct Use |
| |
|
| | These adapter weights must be merged with the base LLaMA-3-8B model using PEFT or Hugging Face’s `PeftModel` wrapper. |
| |
|
| | Example use case: |
| |
|
| | ```python |
| | from transformers import AutoTokenizer, AutoModelForCausalLM |
| | from peft import PeftModel, PeftConfig |
| | |
| | base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B") |
| | model = PeftModel.from_pretrained(base_model, "saramshgautam/lora-llama-8b-github") |
| | tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B") |
| | ``` |
| |
|
| | ### Intended USe |
| |
|
| | - Research in summarization of technical conversations |
| |
|
| | - Augmenting code review and issue tracking pipelines |
| |
|
| | - Studying model adaptation via parameter-efficient fine-tuning |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | - Commercial applications (restricted by Meta’s LLaMA license) |
| |
|
| | - General-purpose conversation or chatbot use (model optimized for summarization) |
| |
|
| | ## Bias, Risks, and Limitations |
| |
|
| | - The model inherits biases from both the base LLaMA-3 model and the GitHub dataset. It may underperform on non-technical content or multilingual issues. |
| |
|
| | ## Recommendations |
| |
|
| | Use only for academic or non-commercial research. Evaluate responsibly before using in production or public-facing tools. |
| |
|
| | ## How to Get Started with the Model |
| |
|
| | See the example in “Direct Use” above. You must separately download the base model from Meta and load the LoRA adapters from this repo. |
| |
|
| | ## Training Details |
| |
|
| | ### Training Data |
| |
|
| | - Source: Hugging Face lewtun/github-issues |
| | - Description: Contains 3,000+ GitHub issues and comments from popular open-source repositories. |
| |
|
| | ## Training Procedure |
| |
|
| | - LoRA with PEFT |
| | - 4-bit quantized training using bitsandbytes |
| | - Mixed precision: bf16 |
| | - Batch size: 8 |
| | - Epochs: 3 |
| | - Optimizer: AdamW |
| |
|
| | ### Evaluation |
| |
|
| | ## Metrics |
| |
|
| | ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-Lsum on a 500-issue test set |
| |
|
| | ## Results |
| |
|
| | | Metric | Score | |
| | | ---------- | ----- | |
| | | ROUGE-1 | 0.706 | |
| | | ROUGE-2 | 0.490 | |
| | | ROUGE-L | 0.570 | |
| | | ROUGE-Lsum | 0.582 | |
| |
|
| | --- |
| |
|
| | ## Environmental Impact |
| |
|
| | - **Hardware Type:** 4×A100 GPUs (university HPC cluster) |
| | - **Training Hours:** ~4 hours |
| | - **Carbon Estimate:** ~10.2 kg CO₂eq |
| | _(estimated via [ML CO2 calculator](https://mlco2.github.io/impact))_ |
| |
|
| | --- |
| |
|
| | ## Citation |
| |
|
| | **APA:** |
| |
|
| | Gautam, S. (2025). _LoRA-LLaMA3-8B-GitHub-Summarizer: Adapter weights for summarizing GitHub issues using LLaMA 3_. Hugging Face. https://huggingface.co/saramshgautam/lora-llama-8b-github |
| |
|
| | **BibTeX:** |
| |
|
| | ```bibtex |
| | @misc{gautam2025lora, |
| | title={LoRA-LLaMA3-8B-GitHub-Summarizer}, |
| | author={Gautam, Saramsh}, |
| | year={2025}, |
| | howpublished={\url{https://huggingface.co/saramshgautam/lora-llama-8b-github}}, |
| | note={Fine-tuned adapter weights using LoRA on Meta-LLaMA-3-8B} |
| | } |
| | ``` |
| |
|
| | --- |
| |
|
| | ## Contact |
| |
|
| | - **Author:** Saramsh Gautam |
| | - **Affiliation:** Louisiana State University |
| | - **Email:** [your email] |
| | - **Hugging Face profile:** [https://huggingface.co/saramshgautam](https://huggingface.co/saramshgautam) |
| |
|
| | --- |
| |
|
| | ## Framework Versions |
| |
|
| | - **PEFT:** 0.15.2 |
| | - **Transformers:** 4.40.0 |
| | - **Bitsandbytes:** 0.41.3 |
| | - **Datasets:** 2.18.0 |
