Raya: Laya multilingual fine-tuned for 3-tier routing

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,110 @@
+---
+license: apache-2.0
+library_name: laya
+pipeline_tag: text-classification
+base_model: convaiinnovations/laya
+datasets:
+  - allenai/WildChat-1M
+language: [en, de, fr, es, it, pt, nl, pl, tr, ru, ar, zh, ja, ko]
+tags: [routing, llm-routing, model-selection, prompt-difficulty, laya, system-one, calibrated-decisions, multilingual]
+---
+
+# Raya — a fast multilingual LLM router (Laya fine-tune)
+
+**Raya picks which model tier should answer a prompt — a small fast model, a mid-tier model, or a
+frontier model — in one ~17 ms forward pass, about 20× faster than a hosted routing API.**
+
+- **Robust to how you phrase the routing question.** Raya scores 80–81% on all three question styles we
+  tested. On the difficulty-score style it beats TypeSafe Jev by ~10 points
+  (**80.3% vs 70.5%**, paired McNemar p < 0.001).
+- **Matches or beats Jev on the minimal routing question in Japanese (90 vs 87), Polish (86 vs 83),
+  Spanish (87 vs 85), German (74 vs 72), English (86 vs 86), and Arabic (73 vs 73).**
+- **Open and self-hostable.** A 300M-parameter model with no per-call cost, served through Laya's
+  Jev-compatible HTTP API.
+
+Raya is [Laya](https://huggingface.co/convaiinnovations/laya)'s multilingual decision model
+(mmBERT-base encoder), fine-tuned end-to-end for 3-tier routing. It is not affiliated with Convai
+Innovations or TypeSafe.
+
+## Quick start
+
+```python
+import laya  # pip install laya  (Raya was built and tested with laya 0.3.7)
+
+raya = laya.Agent("TextCortex/raya", device="cuda")   # or "mps" / "cpu"
+
+ROUTE = {
+    "type": "choice",
+    "instructions": "Route this prompt to a model.",
+    "criteria": {
+        "small_model": "simple requests",
+        "medium_model": "moderately complex requests",
+        "frontier_model": "very hard requests",
+    },
+}
+
+out = raya.system_one({"prompt": "Schreibe eine professionelle E-Mail an einen Kunden …"}, {"route": ROUTE})
+print(out["answers"]["route"])
+# {'choice': 'medium_model', 'probabilities': {'small_model': 0.24, 'medium_model': 0.74, 'frontier_model': 0.03}, ...}
+```
+
+Raya was trained on three routing questions: the minimal choice above, a detailed rubric choice, and a
+3-level difficulty score. Use one of those. Option order does not matter because options were shuffled in
+training. Raya serves through Laya's Jev-compatible HTTP server (`POST /v1/systemone`).
+
+## Benchmark: 3-tier routing on real multilingual prompts
+
+**Test set.** 563 first-turn prompts from WildChat-1M (shards never used for training), 14 languages,
+~40 per language, 75% natural traffic + 25% hard-leaning. Gold tier = the tier two independent blind
+annotators (Claude Opus and Claude Sonnet, same written rubric) **both** assigned; the annotators agree on
+78% of prompts, which is the practical ceiling. Tier mix: 225 small / 317 medium / 21 frontier.
+
+| Question style | Raya | TypeSafe Jev 1.13 | Laya 0.3.7 (stock) | Von 1.1 (stock) |
+|---|---|---|---|---|
+| Difficulty score (3 levels) | **80.3%** | 70.5% | 61.6% | 42.3% |
+| Rubric choice | 81.0% | **84.2%** | 47.1% | 58.8% |
+| Minimal choice | 80.8% | **84.5%** | 55.2% | 54.5% |
+| **Latency (p50)** | **17 ms** (1× GPU, in-process) | ~330 ms (hosted API) | — | — |
+
+Always answering "medium" scores 56.3%.
+
+**Per language, minimal choice** (accuracy %, n = 31–53 per language):
+
+| | ja | pl | es | de | en | ar | it | ko | nl | ru | fr | zh | pt | tr |
+|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
+| Raya | **90** | **86** | **87** | **74** | 86 | 73 | 87 | 88 | 78 | 82 | 78 | 79 | 76 | 67 |
+| Jev | 87 | 83 | 85 | 72 | 86 | 73 | **90** | **91** | **81** | **89** | **86** | **88** | **90** | **82** |
+
+Confusion, minimal choice (rows = gold small/medium/frontier, cols = Raya): `[[169, 56, 0], [30, 279, 8], [0, 14, 7]]`.
+
+**Limitations of these numbers.** On the choice questions Jev is ahead overall by 3–4 points. That gap
+is not statistically significant at these sample sizes (p = 0.07 and 0.13), but it is consistent. Raya is
+weakest in Turkish, Portuguese, and Chinese. Differences of less than ~10 points within a single language
+are within noise. Like Jev, Raya rarely routes to the frontier tier (7 of 21). Gold labels come from LLM
+annotators, not from measured downstream answer quality.
+
+## Training
+
+- **Architecture:** Laya decision model — mmBERT-base encoder + 2-layer transformer decision head with
+  per-option `[MASK]` scoring, initialised from `convaiinnovations/laya` (`multilingual`) and fine-tuned
+  end-to-end.
+- **Data (soft labels from two blind annotators; 50/50 target where they disagreed):** WildChat-1M
+  first-turn prompts (ODC-BY); synthetic hard / hard-looking prompts across the 14 languages (kept only
+  if a blind annotator confirmed the intended tier); additional in-house routing data.
+- **Procedure:** soft-target cross-entropy, options shuffled, AdamW (encoder 2e-5, head 1e-4), batch 32,
+  sequences capped at 512 tokens, bf16, best epoch chosen by validation accuracy (86.7% on 659 held-out
+  validation prompts, minimal choice), per-question temperature fitted on validation. The seed was also
+  chosen on validation only.
+- **Compute:** one NVIDIA RTX A6000, ~6 minutes.
+
+## Intended use and limitations
+
+For choosing among 2–3 model tiers for chat/assistant traffic. Not for safety moderation or for more
+than 3 tiers. Not validated on multi-turn context. Calibrate a confidence threshold on your own traffic
+before auto-escalating.
+
+## Acknowledgements
+
+Laya architecture, runtime, and base checkpoint by Convai Innovations (Apache-2.0). mmBERT by JHU CLSP
+(MIT). WildChat-1M by AI2 (ODC-BY). Jev is a product of TypeSafe AI; its scores come from our own runs
+via its API (September 2026).

encoder/config.json

@@ -0,0 +1,79 @@
+{
+  "architectures": [
+    "ModernBertForMaskedLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 1,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "dtype": "float32",
+  "embedding_dropout": 0.0,
+  "eos_token_id": 1,
+  "global_attn_every_n_layers": 3,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention"
+  ],
+  "local_attention": 128,
+  "mask_token_id": 4,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 0,
+  "position_embedding_type": "sans_pos",
+  "repad_logits_with_grad": false,
+  "rope_parameters": {
+    "full_attention": {
+      "rope_theta": 160000,
+      "rope_type": "default"
+    },
+    "sliding_attention": {
+      "rope_theta": 160000,
+      "rope_type": "default"
+    }
+  },
+  "sep_token_id": 1,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "vocab_size": 256000
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44a55826828b36f2b391364fad88741aa8d13605a4bae99215dab2c74a869b05
+size 1287653720

rl_agent_config.json

@@ -0,0 +1,42 @@
+{
+  "encoder": "jhu-clsp/mmBERT-base",
+  "head_layers": 2,
+  "max_len": 1024,
+  "head_max_len": 256,
+  "max_prefixes": 6,
+  "act_costs": {
+    "escalate": 0.5
+  },
+  "cost_wrong_act": 3.0,
+  "amp_dtype": "bf16",
+  "model_name": "rl-agent",
+  "temperature": [
+    1.0,
+    1.0,
+    1.0
+  ],
+  "temperature_by_options": {
+    "choice:3-5": 1.35,
+    "score:3-5": 1.35
+  },
+  "training": {
+    "updates": 15987,
+    "epochs_completed": 4,
+    "hours": 4.97,
+    "world_size": 1,
+    "fine_tuned_from_checkpoint": true,
+    "routing_finetune": {
+      "rows": 9214,
+      "synthetic_rows": 767,
+      "epochs": 2,
+      "select": "acc",
+      "best_val_criterion": -0.8558,
+      "temperatures": {
+        "Q1_rubric_choice": 1.3,
+        "Q2_minimal_choice": 1.4,
+        "Q3_difficulty_score": 1.35
+      },
+      "seed": 1
+    }
+  }
+}

tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:609d8f4c067cd3950f88594c5a802616cea245823836ef5848ee4fc40aab5b6f
+size 34363188

tokenizer/tokenizer_config.json

@@ -0,0 +1,22 @@
+{
+  "bos_token": "<bos>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<bos>",
+  "eos_token": "<eos>",
+  "extra_special_tokens": {
+    "extra_0": "<start_of_turn>",
+    "extra_1": "<end_of_turn>"
+  },
+  "mask_token": "<mask>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 8192,
+  "pad_token": "<pad>",
+  "padding_side": "right",
+  "sep_token": "<eos>",
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "<unk>"
+}