Update README.md
Browse files
README.md
CHANGED
|
@@ -25,7 +25,8 @@ This is the preview instruct variant.
|
|
| 25 |
|
| 26 |
## post-training info:
|
| 27 |
- this luaudev variant was post-trained with sft on about 1.5 billion tokens of mixed luau specific code, math, q&a and so on. This model supports 4 reasoning efforts: disabled(no reasoning), low, medium, high. Note the model may loop with high reasoning more than normal. Luaudev also supports tool calling and agentic output and inputs yet for this preview the performance is not ideal.
|
| 28 |
-
|
|
|
|
| 29 |
## PLB benchmark results:
|
| 30 |
|
| 31 |
text:
|
|
|
|
| 25 |
|
| 26 |
## post-training info:
|
| 27 |
- this luaudev variant was post-trained with sft on about 1.5 billion tokens of mixed luau specific code, math, q&a and so on. This model supports 4 reasoning efforts: disabled(no reasoning), low, medium, high. Note the model may loop with high reasoning more than normal. Luaudev also supports tool calling and agentic output and inputs yet for this preview the performance is not ideal.
|
| 28 |
+
- the model was post trained with an effective batch size of 32, and with a custom home-made ademamix based 2bit fp8 optimizer for real memory savings.
|
| 29 |
+
- post training was done with a sequence length of 32k, and thus that is the supported max sequence length by the model.
|
| 30 |
## PLB benchmark results:
|
| 31 |
|
| 32 |
text:
|