hugruby commited on
Commit
a1bd9b4
·
verified ·
1 Parent(s): 3b83fcb

Model card: add cross-variant length-budgets table and note

Browse files
Files changed (1) hide show
  1. README.md +11 -0
README.md CHANGED
@@ -116,6 +116,17 @@ python scripts/train.py \
116
  | **Released checkpoint** | **global step 2000** (epoch 0.900) |
117
  | Random seed | 42 |
118
 
 
 
 
 
 
 
 
 
 
 
 
119
  ## Files
120
 
121
  - `adapter_model.safetensors`, `adapter_config.json` — the LoRA adapter (load with PEFT on the base model)
 
116
  | **Released checkpoint** | **global step 2000** (epoch 0.900) |
117
  | Random seed | 42 |
118
 
119
+ **Length budgets across all four variants:**
120
+
121
+ | Variant | max-seq-length | max-completion | max-prompt-tokens |
122
+ |---|---|---|---|
123
+ | **mismatched-wrong** | 7168 | 4096 | 3072 |
124
+ | matched-wrong | 7168 | 4096 | 3072 |
125
+ | no-draft | 7168 | 4096 | disabled (but it is equivalent to 3,072, as all prompts are short and no truncation) |
126
+ | mismatched-correct | 8192 | 4096 | disabled |
127
+
128
+ For a strict apple-to-apple comparison, `mismatched-correct` should have used `--max-seq-length 7168` and `--max-prompt-tokens 3072` like the other three variants; the larger 8,192 with the max-prompt-tokens cap left off was an omission. The effect should be negligible though — only **6 of 8,888** prompts exceed 3,072 tokens (longest 3,317), and for the other 8,882 the run is identical to a 7,168 / 3,072 setup. For those 6 the prompt is left untruncated, but the 4,096 max-completion length is still respected and max-seq-length runs only slightly past 7,168 (at most 3,317 + 4,096 = 7,413, well under 8,192). But to train a precise apple-to-apple version yourself, for mismatched-correct, change `--max-seq-length 8192` to `7168` and add `--max-prompt-tokens 3072`. For the no-draft variant, it is better to add `--max-prompt-tokens 3072` explicitly as well.
129
+
130
  ## Files
131
 
132
  - `adapter_model.safetensors`, `adapter_config.json` — the LoRA adapter (load with PEFT on the base model)