NovatasticRoScript commited on
Commit
56befbc
·
verified ·
1 Parent(s): 9438344

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -17
README.md CHANGED
@@ -30,27 +30,11 @@ base_model:
30
 
31
  **Reasoning-first · Zero benchmark contamination · Trained on a free Colab T4**
32
 
33
-
34
-
35
  ![License](https://img.shields.io/badge/license-CC--BY--4.0-blue)
36
-
37
-
38
-
39
-
40
  ![Base](https://img.shields.io/badge/base-Qwen3--1.7B-orange)
41
-
42
-
43
-
44
-
45
  ![Method](https://img.shields.io/badge/method-GRPO-purple)
46
-
47
-
48
-
49
-
50
  ![Params](https://img.shields.io/badge/params-1.7B-green)
51
 
52
-
53
-
54
  </div>
55
 
56
  ---
@@ -114,6 +98,18 @@ These are the results of the benchmarks for *Atomight-V2.5-1.7B*, evaluated with
114
 
115
  ---
116
 
 
 
 
 
 
 
 
 
 
 
 
 
117
  ## How it compares with other small language models (we recommend verifying it, as the other data from other models came from a third-party sources)
118
 
119
  Scores below for other models are drawn from their respective model cards / technical reports, not re-run by us. Provided for context only — evaluation harnesses and prompt formats differ across labs, so treat this as directional rather than exact.
@@ -178,4 +174,5 @@ Training method — this model was trained using GRPO (Group Relative Policy Opt
178
  journal={arXiv preprint arXiv:2402.03300},
179
  year={2024}
180
  }
181
- ```
 
 
30
 
31
  **Reasoning-first · Zero benchmark contamination · Trained on a free Colab T4**
32
 
 
 
33
  ![License](https://img.shields.io/badge/license-CC--BY--4.0-blue)
 
 
 
 
34
  ![Base](https://img.shields.io/badge/base-Qwen3--1.7B-orange)
 
 
 
 
35
  ![Method](https://img.shields.io/badge/method-GRPO-purple)
 
 
 
 
36
  ![Params](https://img.shields.io/badge/params-1.7B-green)
37
 
 
 
38
  </div>
39
 
40
  ---
 
98
 
99
  ---
100
 
101
+ ## IIfSLM Benchmark Results
102
+
103
+ [IIfSLM](https://huggingface.co/datasets/NovatasticRoScript/IIfSLM-v1) (Intelligence Index for Small Language Models) is an open, contamination-resistant benchmark suite for the 0.5B–4B range — built for the whole small-model community to evaluate against, not exclusive to this model.
104
+
105
+ | Domain | Score | Details |
106
+ |---|---:|---|
107
+ | gsm8krefn | **86.96%** | 260/299 correct · greedy decoding · max_new_tokens=900 |
108
+
109
+ More domains (HumanEval, ARC-Challenge) are in progress. See the [full IIfSLM dataset and methodology notes](https://huggingface.co/datasets/NovatasticRoScript/IIfSLM-v1) — and feel free to run your own model against it too.
110
+
111
+ ---
112
+
113
  ## How it compares with other small language models (we recommend verifying it, as the other data from other models came from a third-party sources)
114
 
115
  Scores below for other models are drawn from their respective model cards / technical reports, not re-run by us. Provided for context only — evaluation harnesses and prompt formats differ across labs, so treat this as directional rather than exact.
 
174
  journal={arXiv preprint arXiv:2402.03300},
175
  year={2024}
176
  }
177
+ ```
178
+