Text Generation
PEFT
Safetensors
English
lora
grpo
dr-grpo
mathematical-reasoning
math
conversational
Instructions to use hugruby/mathstral-7b-mismatched-wrong-drafts with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use hugruby/mathstral-7b-mismatched-wrong-drafts with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mathstral-7B-v0.1") model = PeftModel.from_pretrained(base_model, "hugruby/mathstral-7b-mismatched-wrong-drafts") - Notebooks
- Google Colab
- Kaggle
Model card: add cross-variant length-budgets table and note
Browse files
README.md
CHANGED
|
@@ -116,6 +116,17 @@ python scripts/train.py \
|
|
| 116 |
| **Released checkpoint** | **global step 2000** (epoch 0.900) |
|
| 117 |
| Random seed | 42 |
|
| 118 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 119 |
## Files
|
| 120 |
|
| 121 |
- `adapter_model.safetensors`, `adapter_config.json` — the LoRA adapter (load with PEFT on the base model)
|
|
|
|
| 116 |
| **Released checkpoint** | **global step 2000** (epoch 0.900) |
|
| 117 |
| Random seed | 42 |
|
| 118 |
|
| 119 |
+
**Length budgets across all four variants:**
|
| 120 |
+
|
| 121 |
+
| Variant | max-seq-length | max-completion | max-prompt-tokens |
|
| 122 |
+
|---|---|---|---|
|
| 123 |
+
| **mismatched-wrong** | 7168 | 4096 | 3072 |
|
| 124 |
+
| matched-wrong | 7168 | 4096 | 3072 |
|
| 125 |
+
| no-draft | 7168 | 4096 | disabled (but it is equivalent to 3,072, as all prompts are short and no truncation) |
|
| 126 |
+
| mismatched-correct | 8192 | 4096 | disabled |
|
| 127 |
+
|
| 128 |
+
For a strict apple-to-apple comparison, `mismatched-correct` should have used `--max-seq-length 7168` and `--max-prompt-tokens 3072` like the other three variants; the larger 8,192 with the max-prompt-tokens cap left off was an omission. The effect should be negligible though — only **6 of 8,888** prompts exceed 3,072 tokens (longest 3,317), and for the other 8,882 the run is identical to a 7,168 / 3,072 setup. For those 6 the prompt is left untruncated, but the 4,096 max-completion length is still respected and max-seq-length runs only slightly past 7,168 (at most 3,317 + 4,096 = 7,413, well under 8,192). But to train a precise apple-to-apple version yourself, for mismatched-correct, change `--max-seq-length 8192` to `7168` and add `--max-prompt-tokens 3072`. For the no-draft variant, it is better to add `--max-prompt-tokens 3072` explicitly as well.
|
| 129 |
+
|
| 130 |
## Files
|
| 131 |
|
| 132 |
- `adapter_model.safetensors`, `adapter_config.json` — the LoRA adapter (load with PEFT on the base model)
|