Add ONNX export for transformers.js

README.md

@@ -0,0 +1,121 @@
+---
+license: other
+license_name: lfm1.0
+license_link: https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE
+base_model: LiquidAI/LFM2.5-Encoder-350M-Prompt-Router
+base_model_relation: quantized
+library_name: transformers.js
+pipeline_tag: zero-shot-classification
+tags:
+- onnx
+- transformers.js
+- lfm2
+- quantized
+language:
+- en
+- de
+- es
+- fr
+- it
+- nl
+- pl
+- pt
+- ar
+- hi
+- ja
+- ru
+- tr
+- vi
+- zh
+---
+
+# LFM2.5-Encoder-350M-Prompt-Router-ONNX
+
+ONNX export of [`LiquidAI/LFM2.5-Encoder-350M-Prompt-Router`](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router), quantized to run **fully in the browser** through
+[transformers.js](https://github.com/huggingface/transformers.js). No inference server: the weights are
+fetched once, cached, and every forward pass happens in the tab.
+
+A zero-shot prompt router. Categories are ordinary prose supplied at call time — nothing is
+trained or cached per label set. One bidirectional pass over the category list *and* the text
+scores every category at once.
+
+All credit for the model itself goes to [Liquid AI](https://huggingface.co/LiquidAI). This repository
+contains only a re-export; the weights are unchanged apart from quantization, and the original
+[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M-Prompt-Router/blob/main/LICENSE) applies.
+
+Tooling, demo and the export pipeline: <https://github.com/kucukkanat/lfm-encoders>
+
+## Files
+
+| dtype | File | Size |
+| --- | --- | --: |
+| `q8` | `onnx/model_quantized.onnx` | 357 MB |
+| `q4` | `onnx/model_q4.onnx` | 449 MB |
+
+The graph takes `input_ids` + `attention_mask`, is dynamic in batch and sequence, and returns
+`token_proj` and `rule_proj`.
+
+## Usage
+
+```js
+import { AutoTokenizer, PreTrainedModel, Tensor } from "@huggingface/transformers";
+
+const id = "kucukkanat/LFM2.5-Encoder-350M-Prompt-Router-ONNX";
+const tokenizer = await AutoTokenizer.from_pretrained(id);
+const model = await PreTrainedModel.from_pretrained(id, { dtype: "q8" });
+
+const { input_ids } = tokenizer("some text");
+const out = await model({
+  input_ids,
+  attention_mask: new Tensor("int64", new BigInt64Array(input_ids.dims[1]).fill(1n), input_ids.dims),
+});
+```
+
+`PreTrainedModel` rather than `AutoModel` is deliberate: this is a plain "feed the named inputs, read the
+named outputs" session, not one of transformers.js's built-in architectures.
+
+### Prompt format
+
+Both projection towers expect one string laid out exactly like this — the model was trained on it and the
+character arithmetic that locates each label depends on it byte for byte:
+
+```
+Categories:
+- label one
+- label two
+
+Text:
+<the text>
+```
+
+`token_proj` is the query tower and `rule_proj` the key tower, both 256-d and emitted **per token**. Pool
+the tokens covering each label to get its vector. Pooling after projecting is exact rather than an
+approximation: both towers are affine, and an affine map commutes with a mean — which is what keeps one
+static graph usable for any number of labels.
+
+Scoring is `cosine between the L2-normalised pooled towers, scaled by a learned temperature, then a softmax across labels`.
+
+[`@lfm-encoder/tasks`](https://github.com/kucukkanat/lfm-encoders) implements all of this, including the
+character-offset reconstruction transformers.js does not provide.
+
+
+## Accuracy
+
+Measured from JavaScript against the fp32 PyTorch reference. Δ is the largest absolute difference in a
+final probability.
+
+| dtype | max Δ | mean Δ | top-1 flips (4 cases) |
+| --- | --: | --: | --: |
+| `fp32` | 6.4e-5 | 1.6e-5 | 0 |
+| `q8` | 0.0910 | 0.0230 | 0 |
+| `q4` | 0.1221 | 0.0309 | 0 |
+
+## Notes
+
+- `q8` is smaller on disk but uses **more** browser RAM than fp32 and runs slower: onnxruntime's WASM
+  kernels compute in float, so quantized weights are unpacked at session load. Quantization here buys
+  download size, not speed or memory.
+- Budget roughly 1.5 GB of RAM per resident model, and expect a tab to hold its high-water mark until
+  reloaded.
+- `fp16` / `q4f16` are deliberately absent: RMSNorm's variance overflows fp16 on this architecture and
+  every hidden state collapses to zeros.

config.json

@@ -0,0 +1,54 @@
+{
+  "model_type": "lfm2-bidirectional",
+  "architectures": [
+    "TwoTowerGraph"
+  ],
+  "task": "zero-shot-routing",
+  "source_model": "LiquidAI/LFM2.5-Encoder-350M-Prompt-Router",
+  "hidden_size": 1024,
+  "vocab_size": 65536,
+  "real_vocab_size": 64402,
+  "num_hidden_layers": 16,
+  "layer_types": [
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv",
+    "full_attention",
+    "conv"
+  ],
+  "max_position_embeddings": 128000,
+  "transformers.js_config": {
+    "kv_cache_dtype": null,
+    "use_external_data_format": false
+  },
+  "onnx": {
+    "inputs": [
+      "input_ids",
+      "attention_mask"
+    ],
+    "outputs": [
+      "token_proj",
+      "rule_proj"
+    ]
+  },
+  "head": {
+    "kind": "cosine",
+    "normalize": true,
+    "scale": 1.3714938163757324,
+    "bias": -0.2723352313041687,
+    "activation": "softmax",
+    "prefix_heading": "Categories",
+    "proj_dim": 256
+  }
+}

onnx/model_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac624f7a15ee59888603170573aa39f16268ccdd7d86117f60737f45cc86840a
+size 448954295

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:353b79e0052f80bd4049542f55a487c6d418a79bc1a2e02f28e6c22f432afdc3
+size 356839915

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|startoftext|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "is_local": false,
+  "mask_token": "<|mask|>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<|pad|>",
+  "tokenizer_class": "TokenizersBackend"
+}