Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,56 @@ datasets:
|
|
| 3 |
- XenArcAI/MathX-5M
|
| 4 |
base_model:
|
| 5 |
- google/gemma-3-1b-it
|
| 6 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
- XenArcAI/MathX-5M
|
| 4 |
base_model:
|
| 5 |
- google/gemma-3-1b-it
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# Model Card: Parveshiiii/M1-MathX
|
| 9 |
+
|
| 10 |
+
## Model Details
|
| 11 |
+
- **Model Name:** Parveshiiii/M1-MathX
|
| 12 |
+
- **Base Architecture:** Gemma (1B parameters)
|
| 13 |
+
- **Model Type:** Causal Language Model (text-generation)
|
| 14 |
+
- **Training Framework:** Hugging Face Transformers
|
| 15 |
+
- **Precision:** fp16
|
| 16 |
+
- **Attention Mechanism:** Hybrid sliding-window and full attention layers
|
| 17 |
+
- **Tokenizer:** Gemma tokenizer (vocab size 262,144)
|
| 18 |
+
|
| 19 |
+
## Intended Use
|
| 20 |
+
- Designed for mathematical reasoning tasks, including problem solving, equation manipulation, and step-by-step derivations.
|
| 21 |
+
- Suitable for educational contexts, math tutoring, and research experiments in reasoning alignment.
|
| 22 |
+
- Not intended for general-purpose conversation or sensitive domains outside mathematics.
|
| 23 |
+
|
| 24 |
+
## Training Data
|
| 25 |
+
- **Dataset:** MathX (curated mathematical reasoning dataset)
|
| 26 |
+
- **Samples Used:** ~300
|
| 27 |
+
- **Training Steps:** 50
|
| 28 |
+
- **Method:** GRPO (Group Relative Policy Optimization) fine-tuning
|
| 29 |
+
- **Objective:** Reinforcement-style alignment for improved reasoning clarity and correctness.
|
| 30 |
+
|
| 31 |
+
## Performance
|
| 32 |
+
- Demonstrated strong performance on small-scale math problems and symbolic reasoning tasks.
|
| 33 |
+
- Early benchmarks suggest improved accuracy compared to the base Gemma 1B model on math-specific datasets.
|
| 34 |
+
- Requires formal evaluation on GSM8K, MATH, and other benchmarks for quantitative comparison.
|
| 35 |
+
|
| 36 |
+
## Limitations
|
| 37 |
+
- Small dataset and limited training steps mean coverage is narrow.
|
| 38 |
+
- May overfit to MathX patterns and fail on broader or more complex problems.
|
| 39 |
+
- Not guaranteed to generalize outside mathematical reasoning.
|
| 40 |
+
- As a 1B model, capacity is limited compared to larger LLMs.
|
| 41 |
+
|
| 42 |
+
## Ethical Considerations
|
| 43 |
+
- Intended for safe educational use.
|
| 44 |
+
- Should not be deployed in high-stakes environments without further validation.
|
| 45 |
+
- Outputs may contain errors; human oversight is required.
|
| 46 |
+
|
| 47 |
+
## Citation
|
| 48 |
+
If you use this model, please cite as:
|
| 49 |
+
```
|
| 50 |
+
@misc{Parvesh2025M1MathX,
|
| 51 |
+
author = {Parvesh Rawal},
|
| 52 |
+
title = {Parveshiiii/M1-MathX: A Gemma-1B model fine-tuned on MathX with GRPO},
|
| 53 |
+
year = {2025},
|
| 54 |
+
howpublished = {\url{https://huggingface.co/Parveshiiii/M1-MathX}}
|
| 55 |
+
}
|
| 56 |
+
```
|
| 57 |
+
|
| 58 |
+
---
|