leehack commited on
Commit
770c4c2
·
verified ·
1 Parent(s): cbdf43b

Update results for the renamed held-out command

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -36,11 +36,12 @@ Top intent correct on the example's 48 development commands, which chose the che
36
 
37
  | Head | Development | Held-out |
38
  | --- | --- | --- |
 
39
  | Laya base head, llamadart with `laya-Q8_0.gguf` on Metal | 28 | 18 |
40
  | This head, PyTorch F32 encoder | 46 | 27 |
41
- | This head, llamadart with `laya-Q8_0.gguf` on Metal | 46 | 26 |
42
 
43
- In the example's `bin/bench.dart` with its 0.3 confidence gate, the bar shows the right intent for 25 held-out commands, the wrong one for 5, and stays plain for 2. Retraining with other seeds gave 23 to 27 held-out commands right in PyTorch, so a single run is noisy at this size.
44
 
45
  ## Limitations
46
 
 
36
 
37
  | Head | Development | Held-out |
38
  | --- | --- | --- |
39
+ | Laya base head, PyTorch F32 encoder | 29 | 18 |
40
  | Laya base head, llamadart with `laya-Q8_0.gguf` on Metal | 28 | 18 |
41
  | This head, PyTorch F32 encoder | 46 | 27 |
42
+ | This head, llamadart with `laya-Q8_0.gguf` on Metal | 46 | 27 |
43
 
44
+ In the example's `bin/bench.dart` with its 0.3 confidence gate, the bar shows the right intent for 26 held-out commands, the wrong one for 4, and stays plain for 2. The notebook's other seeds kept heads with up to four fewer held-out commands right in PyTorch, so a single run is noisy at this size.
45
 
46
  ## Limitations
47