Update results for the renamed held-out command
Browse files
README.md
CHANGED
|
@@ -36,11 +36,12 @@ Top intent correct on the example's 48 development commands, which chose the che
|
|
| 36 |
|
| 37 |
| Head | Development | Held-out |
|
| 38 |
| --- | --- | --- |
|
|
|
|
| 39 |
| Laya base head, llamadart with `laya-Q8_0.gguf` on Metal | 28 | 18 |
|
| 40 |
| This head, PyTorch F32 encoder | 46 | 27 |
|
| 41 |
-
| This head, llamadart with `laya-Q8_0.gguf` on Metal | 46 |
|
| 42 |
|
| 43 |
-
In the example's `bin/bench.dart` with its 0.3 confidence gate, the bar shows the right intent for
|
| 44 |
|
| 45 |
## Limitations
|
| 46 |
|
|
|
|
| 36 |
|
| 37 |
| Head | Development | Held-out |
|
| 38 |
| --- | --- | --- |
|
| 39 |
+
| Laya base head, PyTorch F32 encoder | 29 | 18 |
|
| 40 |
| Laya base head, llamadart with `laya-Q8_0.gguf` on Metal | 28 | 18 |
|
| 41 |
| This head, PyTorch F32 encoder | 46 | 27 |
|
| 42 |
+
| This head, llamadart with `laya-Q8_0.gguf` on Metal | 46 | 27 |
|
| 43 |
|
| 44 |
+
In the example's `bin/bench.dart` with its 0.3 confidence gate, the bar shows the right intent for 26 held-out commands, the wrong one for 4, and stays plain for 2. The notebook's other seeds kept heads with up to four fewer held-out commands right in PyTorch, so a single run is noisy at this size.
|
| 45 |
|
| 46 |
## Limitations
|
| 47 |
|