Add the command-tuned Laya head

README.md

@@ -0,0 +1,53 @@
+---
+license: apache-2.0
+base_model: convaiinnovations/laya
+library_name: llamadart
+tags:
+  - laya
+  - decision-model
+  - intent-classification
+  - safetensors
+---
+
+# Laya command head
+
+A decision head for [Laya](https://huggingface.co/convaiinnovations/laya), fine-tuned to read the intent of a command typed into an app. It is used by llamadart's [`laya_command_bar` example](https://github.com/leehack/llamadart/tree/main/example/laya_command_bar) through `DecisionEngine`.
+
+`laya-head-commands.safetensors` holds the 36 head tensors under Laya's PyTorch names, in F32, with no encoder tensors. Its `laya.config` metadata is the unmodified `rl_agent_config.json` of `convaiinnovations/laya` at revision `1c5edc17a7acd8701df6fc341c0d179f1c62c982`, so `DecisionEngine.load` needs no `configPath`. Pair it with a Laya ModernBERT backbone GGUF, such as `laya-Q8_0.gguf` from [`fr0stbit3/laya-gguf`](https://huggingface.co/fr0stbit3/laya-gguf).
+
+```dart
+final decisions = await DecisionEngine.load(
+  engine, // a LlamaEngine with the Laya backbone GGUF loaded
+  headPath: 'laya-head-commands.safetensors',
+);
+```
+
+The head was trained on one `choice` question, with the typed text as the state: "What does the user want to do with this text typed into the app?", over eight options: `search`, `task`, `event`, `reminder`, `message`, `calculate`, `ask` and `settings`, each with the one-line criterion in the example's `lib/src/intents.dart`.
+
+## Training
+
+- **Base:** the head of `convaiinnovations/laya` at `1c5edc17`, with the encoder frozen. Only `head.*`, `type_emb.*` and `scorer.*` were trained; the act head and the temperatures are unchanged.
+- **Data:** 3,063 labelled commands from the example's `bin/make_dataset.dart`: 48 seed commands, 1,346 from templates, and 1,669 generated by Qwen3.8-27B (Q4_K_M) with `bin/generate_commands.dart` and kept only where the same model, asked again with `bin/verify_commands.dart`, gave the same intent (1,949 generated). Commands that normalize to a development or held-out command were dropped. Targets are one-hot.
+- **Recipe:** 60 epochs; AdamW at lr 3e-4, weight decay 0.01; 50 warmup steps, then cosine decay; batch 32; three seeds. The epoch with the best accuracy on the 48 development commands was kept: seed 0, epoch 29. This is the example's `training/laya_head_tuning.ipynb`.
+
+## Results
+
+Top intent correct on the example's 48 development commands, which chose the checkpoint and the gate, and 32 held-out commands, which did not:
+
+| Head | Development | Held-out |
+| --- | --- | --- |
+| Laya base head, llamadart with `laya-Q8_0.gguf` on Metal | 28 | 18 |
+| This head, PyTorch F32 encoder | 46 | 27 |
+| This head, llamadart with `laya-Q8_0.gguf` on Metal | 46 | 26 |
+
+In the example's `bin/bench.dart` with its 0.3 confidence gate, the bar shows the right intent for 25 held-out commands, the wrong one for 5, and stays plain for 2. Retraining with other seeds gave 23 to 27 held-out commands right in PyTorch, so a single run is noisy at this size.
+
+## Limitations
+
+- Trained only on the question above and its eight options. Other questions and option sets have not been evaluated with this head.
+- Short English commands only, like the base model.
+- The example's EmbeddingGemma nearest-example reader and small LLM readers are more accurate on the same commands; this head is a fine-tuning example, not the best intent reader.
+
+## License
+
+Apache-2.0, like the base model. This is a modified version of the `convaiinnovations/laya` head: the weights listed under Training were fine-tuned on the command data.

laya-head-commands.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95c0775b1d1b9ec7e0644aa31e28f9980805b4d5c8b4939fc177c7991f0c24f8
+size 106052824