Update README.md
Browse files
README.md
CHANGED
|
@@ -30,27 +30,11 @@ base_model:
|
|
| 30 |
|
| 31 |
**Reasoning-first · Zero benchmark contamination · Trained on a free Colab T4**
|
| 32 |
|
| 33 |
-
|
| 34 |
-
|
| 35 |

|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |

|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |

|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |

|
| 51 |
|
| 52 |
-
|
| 53 |
-
|
| 54 |
</div>
|
| 55 |
|
| 56 |
---
|
|
@@ -114,6 +98,18 @@ These are the results of the benchmarks for *Atomight-V2.5-1.7B*, evaluated with
|
|
| 114 |
|
| 115 |
---
|
| 116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
## How it compares with other small language models (we recommend verifying it, as the other data from other models came from a third-party sources)
|
| 118 |
|
| 119 |
Scores below for other models are drawn from their respective model cards / technical reports, not re-run by us. Provided for context only — evaluation harnesses and prompt formats differ across labs, so treat this as directional rather than exact.
|
|
@@ -178,4 +174,5 @@ Training method — this model was trained using GRPO (Group Relative Policy Opt
|
|
| 178 |
journal={arXiv preprint arXiv:2402.03300},
|
| 179 |
year={2024}
|
| 180 |
}
|
| 181 |
-
```
|
|
|
|
|
|
| 30 |
|
| 31 |
**Reasoning-first · Zero benchmark contamination · Trained on a free Colab T4**
|
| 32 |
|
|
|
|
|
|
|
| 33 |

|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |

|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |

|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |

|
| 37 |
|
|
|
|
|
|
|
| 38 |
</div>
|
| 39 |
|
| 40 |
---
|
|
|
|
| 98 |
|
| 99 |
---
|
| 100 |
|
| 101 |
+
## IIfSLM Benchmark Results
|
| 102 |
+
|
| 103 |
+
[IIfSLM](https://huggingface.co/datasets/NovatasticRoScript/IIfSLM-v1) (Intelligence Index for Small Language Models) is an open, contamination-resistant benchmark suite for the 0.5B–4B range — built for the whole small-model community to evaluate against, not exclusive to this model.
|
| 104 |
+
|
| 105 |
+
| Domain | Score | Details |
|
| 106 |
+
|---|---:|---|
|
| 107 |
+
| gsm8krefn | **86.96%** | 260/299 correct · greedy decoding · max_new_tokens=900 |
|
| 108 |
+
|
| 109 |
+
More domains (HumanEval, ARC-Challenge) are in progress. See the [full IIfSLM dataset and methodology notes](https://huggingface.co/datasets/NovatasticRoScript/IIfSLM-v1) — and feel free to run your own model against it too.
|
| 110 |
+
|
| 111 |
+
---
|
| 112 |
+
|
| 113 |
## How it compares with other small language models (we recommend verifying it, as the other data from other models came from a third-party sources)
|
| 114 |
|
| 115 |
Scores below for other models are drawn from their respective model cards / technical reports, not re-run by us. Provided for context only — evaluation harnesses and prompt formats differ across labs, so treat this as directional rather than exact.
|
|
|
|
| 174 |
journal={arXiv preprint arXiv:2402.03300},
|
| 175 |
year={2024}
|
| 176 |
}
|
| 177 |
+
```
|
| 178 |
+
|