| --- |
| base_model: sentence-transformers/all-MiniLM-L6-v2 |
| library_name: peft |
| license: mit |
| tags: |
| - lora |
| - peft |
| - code |
| - programming |
| - software |
| - domain-adaptation |
| - sentence-embeddings |
| language: |
| - en |
| --- |
| |
| # Code LoRA Adapter for DomainEmbedder-v2.6 |
|
|
| Domain-specific LoRA adapter for code/programming text embeddings. |
|
|
| ## Model Details |
|
|
| | Property | Value | |
| |----------|-------| |
| | **Base Model** | sentence-transformers/all-MiniLM-L6-v2 | |
| | **Parent System** | DomainEmbedder-v2.6 | |
| | **Domain** | Code / Programming | |
| | **LoRA Rank** | 16 | |
| | **LoRA Alpha** | 32 | |
| | **Target Modules** | query, value | |
| | **Trainable Params** | 147,456 (0.645%) | |
|
|
| ## Training Data |
|
|
| Trained on 40,000 code-related text pairs from: |
| - Code Alpaca |
| - MBPP (Mostly Basic Python Problems) |
| - Code Contests |
| - Python Instructions |
|
|
| ## Training Configuration |
|
|
| | Parameter | Value | |
| |-----------|-------| |
| | Epochs | 3 | |
| | Batch Size | 32 | |
| | Learning Rate | 2e-4 | |
| | Loss | Contrastive (InfoNCE) | |
| | Best Val Loss | 0.0039 | |
|
|
| ## Usage |
|
|
| This adapter is part of the DomainEmbedder-v2.6 system. It is selected automatically by the RL policy when code-related content is detected. |
|
|
| ```python |
| from peft import PeftModel |
| from transformers import AutoModel |
| |
| # Load base encoder |
| base_encoder = AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2') |
| |
| # Apply code LoRA |
| code_model = PeftModel.from_pretrained(base_encoder, 'path/to/code_lora') |
| ``` |
|
|
| ## Author |
|
|
| **Zain Asad** |
|
|
| ## License |
|
|
| MIT License |
|
|
| ## Framework Versions |
|
|
| - PEFT 0.18.1 |
| - Transformers 4.x |
| - PyTorch 2.x |
|
|
