Ephraimmm/Qween_lora_weights

Qween LoRA Weights

Overview

This repository contains a LoRA (Low-Rank Adaptation) adapter for the Qwen3-14B causal language model, fine-tuned by Ephraimmm on a dataset of 10,000+ examples of Nigerian/West African Pidgin English. The adapter was trained using Unsloth for accelerated, memory-efficient fine-tuning together with Hugging Face's TRL library. It adapts the base Qwen3 model's conversational ability toward understanding and generating text in Pidgin English.

Training Details

Detail	Value
Base model	unsloth/qwen3-14b-unsloth-bnb-4bit (Qwen3-14B, 4-bit quantized)
Adapter type	LoRA (PEFT)
LoRA rank (r)	32
LoRA alpha	32
LoRA dropout	0
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Bias	none
Task type	Causal LM
Training framework	Unsloth + Hugging Face TRL/PEFT
Training data	10,000+ Pidgin English examples

Note: exact training step/epoch counts and loss curves are not included in this repository (no trainer_state.json was published), so they are omitted here rather than estimated.

Intended Use

This adapter is intended for:

• Experimentation with instruction-following / chat generation in Nigerian Pidgin English.
• Research into low-resource / under-represented African language adaptation of large language models.
• As a base for further fine-tuning or evaluation on Pidgin English NLP tasks.

It is not intended for high-stakes decision-making, medical, legal, or safety-critical applications.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/qwen3-14b-unsloth-bnb-4bit"
adapter_id = "Ephraimmm/Qween_lora_weights"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [{"role": "user", "content": "How you dey today?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Alternatively, load with Unsloth for faster inference:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Ephraimmm/Qween_lora_weights",
    max_seq_length=4096,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

Limitations

• This is a LoRA adapter, not a standalone model — it requires the base unsloth/qwen3-14b-unsloth-bnb-4bit weights to run.
• No quantitative evaluation metrics (e.g., perplexity, benchmark scores) are published for this adapter; performance claims beyond what is documented here should not be assumed.
• As with the base Qwen3 model, outputs may contain inaccuracies, bias, or culturally inappropriate content, particularly given the informal/low-resource nature of Pidgin English training data.
• Training data composition and licensing details beyond "10,000+ Pidgin English examples" are not publicly documented in this repository.

Author

Developed by Ephraimmm.