| --- |
| library_name: transformers |
| tags: [] |
| --- |
| # Model Card for `zay25/MNLP_M3_quantized_model` |
| |
| This model is a quantized version of a multiple-choice question answering (MCQA) model fine-tuned on STEM datasets. It uses Activation-aware Weight Quantization (AWQ) to reduce model size and VRAM usage while preserving strong performance. The model is well-suited for memory- and latency-constrained environments. |
| |
| --- |
| |
| ## Model Details |
| |
| - **Developed by**: Zeineb Mellouli (EPFL, CS-552 Project) |
| - **Base model**: `hssawhney/Best-Performing-Model` (Qwen3-0.6B-Base) |
| - **Quantization**: AWQ (4-bit weights, 16-bit activations) |
| - **Architecture**: Transformer-based Causal Language Model |
| - **Language**: English |
| - **License**: Apache 2.0 |
| |
| --- |
| |
| ## Uses |
| |
| ### Direct Use |
| |
| This model is intended for multiple-choice question answering (MCQA) tasks, particularly in science, math, and engineering education datasets. It is optimized for inference on GPUs with limited VRAM (e.g., A10, T4, or laptop GPUs). |
| |
| ### Out-of-Scope Use |
| |
| - Not intended for open-ended or dialog generation |
| - Not suitable for high-stakes decision-making or critical applications without human oversight |
| |
| |
| ## Training Details |
| |
| - **Quantization method**: Post-training quantization using [AWQ (Activation-aware Weight Quantization)](https://github.com/mit-han-lab/awq) via the `awq` library |
| - **Base model**: `hssawhney/Best-Performing-Model`, fine-tuned on MCQA-style reasoning tasks |
| - **Quantization configuration**: |
| - 4-bit weights (`w_bit = 4`) |
| - Group size: 64 |
| - Per-channel zero point: enabled |
| - **Calibration dataset**: 512 samples from `hssawhney/Reasoning-Dataset` |
|
|
| --- |
|
|
| ## How to Use |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model = AutoModelForCausalLM.from_pretrained("zay25/MNLP_M3_quantized_model", trust_remote_code=True) |
| tokenizer = AutoTokenizer.from_pretrained("zay25/MNLP_M3_quantized_model") |
| |
