code-daemon-denoise-v1

A bilingual (EN + RU) word filter: given one word form, it answers whether that word is a meaningful technical term worth keeping in a search vocabulary, or ballast to drop.

It is deliberately small and one-purpose. A frozen multilingual-e5-small encoder produces a 384-dim vector, and a single trained affine turns that vector into P(keep). No fine-tuning of the encoder, no classification head with its own weights to load — the entire learned decision is 384 numbers and a bias, shipped as a 6 KB JSON file.

That buys throughput: ~800 words/sec on a CPU core, ~17 800/sec on a laptop GPU.

emb = session.run(None, {"input_ids": ids, "attention_mask": mask})[0]   # [B, 384] pooled + L2
p_keep = 1 / (1 + np.exp(-(emb @ w + b)))                               # the whole classifier

---

1. What it is for

Vocabulary hygiene. Harvest every word form out of a codebase — identifiers, doc prose, comments, commit messages — and most of what you get is not worth indexing: inflected function words, chopped identifier fragments, transliteration noise, boilerplate. Keeping them inflates a search vocabulary and dilutes term statistics; dropping them by frequency alone throws away rare-but-real technical terms, which are exactly the ones worth searching for.

This model makes that call per word, in both English and Russian, at a rate that keeps up with a full-repository scan.

Suited to

• Filtering a harvested vocabulary before indexing.
• Any per-token keep/drop decision over short, single-word inputs.
• Mixed EN/RU corpora — including Cyrillic identifiers and comments.

Not suited to

• Sentences or phrases. Inputs are single word forms; the sequence budget is 40 tokens.
• Languages outside Latin/Cyrillic scripts — the vocabulary was pruned to those on purpose.
• Domain term-vs-stopword calls outside software; the label set is technical-corpus flavoured.

---

2. Architecture

Encoder	intfloat/multilingual-e5-small — XLM-RoBERTa, frozen, unchanged
Embedding dim	384, mean-pooled and L2-normalised inside the graph
Vocabulary	142k pieces, pruned from 250k by character class (Latin + Cyrillic + punctuation)
Classifier	one affine: P(keep) = sigmoid(w·e + b), w ∈ ℝ³⁸⁴
Sequence	40 tokens, batch 64
Inputs	input_ids, attention_mask
Output	[batch, 384] — pooled, normalised, ready for the dot product

Two decisions that make it small

The encoder is frozen. The head is a logistic regression fitted on top of fixed embeddings, then folded — its StandardScaler and the LR coefficients are multiplied out into a single (w, b) pair. There is no scikit-learn at inference, and no second model to keep in sync: the decision boundary is a dot product you can apply in any language.

The vocabulary is pruned by script. Cutting the 250k multilingual SentencePiece table to the Latin + Cyrillic + punctuation pieces removes 43% of the rows, and the embedding table is most of this model's weight. The pruned-vocab id remap is baked into the graph as a Gather at the input, so callers still feed ordinary SentencePiece ids and never see the mapping. INT8 weights drop from ~121 MB to **76 MB** — lossless for the two languages it targets, because nothing outside those scripts was reachable anyway.

The "vocab: " prefix

Words are embedded with a fixed "vocab: " prefix. The head was trained on prefixed embeddings, so reproduce the prefix for standalone use or the decision boundary will not line up.

---

3. How it was made

The encoder is frozen — exported to ONNX with mean-pooling and L2-norm fused into the graph, its embedding table pruned to the kept character classes, and quantized to INT8. The decision on top of it is a logistic regression fitted over those fixed embeddings and folded into a single affine, which is why the whole classifier ships as 384 numbers and a bias rather than as a second model.

strip_threshold (default 0.95) sets where you cut. It is high on purpose: dropping a real technical term is the expensive error, keeping a bit of ballast is not.

---

4. Speed

Measured on one laptop: Intel Core Ultra 9 275HX / NVIDIA RTX 5060 Laptop, batch 64 × seq 40.

lane	per batch	throughput	per word
TensorRT FP16, RTX 5060 Laptop	3.60 ms	17 790 words/s	0.056 ms
OpenVINO INT8, iGPU (OV 2026.3)	56.1 ms	1 140 words/s	0.88 ms
OpenVINO INT8, CPU (OV 2026.3)	75-83 ms	770-850 words/s	1.18-1.30 ms
OpenVINO INT4, NPU (OV 2026.3)	57.8 ms	277 words/s	3.61 ms
ONNX Runtime FP32, CPU	188 ms	341 words/s	2.93 ms

The NPU number is per batch 16, not 64 — it is a low-batch part, so its per-word cost is the highest of the three even though its per-batch latency looks similar to the CPU's. Running all three Intel devices at once yields ~87 % of the sum of their solo rates (they share one memory controller), so a host with no discrete GPU can still denoise ~2 000 words/s.

The INT8 CPU lane is the intended default — 800 words/sec is enough to filter a repository's whole harvested vocabulary in seconds without touching a GPU, and it is 2.3× the unquantized ONNX path. The GPU lane exists for hosts that have spare VRAM anyway.

---

5. Standalone use

import json, numpy as np, onnxruntime as ort, sentencepiece as spm

sp   = spm.SentencePieceProcessor(model_file="sentencepiece.bpe.model")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
head = json.load(open("denoise_head.json"))            # {dim, w[384], b, strip_threshold}
w, b, thr = np.array(head["w"], np.float32), head["b"], head["strip_threshold"]

def p_keep(words, max_len=40):
    toks = [[2, *sp.encode("vocab: " + x)[: max_len - 2], 3] for x in words]   # bos … eos
    L    = max(len(t) for t in toks)
    ids  = np.array([t + [0] * (L - len(t)) for t in toks], dtype=np.int64)    # pad = 0
    mask = (ids != 0).astype(np.int64)
    emb  = sess.run(None, {"input_ids": ids, "attention_mask": mask})[0]       # [B, 384]
    return 1.0 / (1.0 + np.exp(-(emb @ w + b)))

scores = p_keep(["mutex", "tensorrt", "пожалуйста", "asdfgh"])
keep   = scores >= thr

The ONNX bakes in the fairseq +1 id offset and the pruned-vocab remap, so feed raw SentencePiece ids — do not remap them yourself.

---

6. Evaluation

Re-measured 2026-09-07 on the shipped INT8 engine against the frozen held-out set (224 words, 95 keep / 129 drop), at strip_threshold = 0.95:

metric	value
SAFE (keep) F1	0.70
BALLAST (drop) F1	0.82
Strip precision	0.89
Strip recall	0.77
Technical words wrongly dropped	12 / 95

Strip precision is the number to watch if you tune the threshold: it says how often a word the model drops really was ballast. The count of technical words wrongly dropped is the one to watch if you care about the expensive direction.

Head calibration (2026-09-07). The head shipped before this date was fitted against an earlier build of the encoder, and its scores saturated: 174 of the 224 held-out words landed at P < 0.01 or P > 0.99, so strip_threshold decided only twelve of them and every useful operating point sat above t = 0.99 — out of reach of the documented knob. The head has been temperature-scaled (w and b divided by T = 2.13). That is a monotone transform, so the model's ranking, its ROC and all SAFE metrics are unchanged by construction; only the probability scale moves, which puts the threshold back in charge. At the default 0.95 this halves the technical words wrongly dropped (24 → 12) for 15 fewer ballast words removed (114 → 99).

On the numbers previously published here (SAFE F1 0.79 / strip precision 0.88). They are not reproducible from the artifacts in this repository, and re-measuring is what surfaced the saturation above. Two independent implementations — the daemon's own harness and a standalone script — agree on the table above to the row. The earlier figures appear to belong to an encoder build that predates the vocabulary prune and is no longer published; they are corrected rather than defended.

---

7. What is in this repo

• OpenVINO INT8 — code-daemon-denoise-v1-s_ov2026.3_{cpu,igpu_lnl}_int8_b64_s40.{xml,bin} — the default lane (CPU) and an Intel iGPU build.
• OpenVINO INT4, NPU — code-daemon-denoise-v1-s_ov2026.3_npu_int4_b16_s40.{xml,bin} — weight-only INT4 at batch 16 for Intel NPUs.
• TensorRT FP16 — code-daemon-denoise-v1-s_{win_x64,linux_x64}_trt11.0_sm_120.engine.
• TVM Vulkan — code-daemon-denoise-v1_{win_x64,linux_x64}_tvm0.25_vulkan.{dll,so} — GPU fallback for non-NVIDIA hardware.
• Head — denoise_head.json. Required: the ONNX alone emits embeddings, not a decision.
• Tokenizer — sentencepiece.bpe.model, tokenizer_config.json.
• ONNX — model.onnx, FP32, pruned, with mean-pool + L2-norm + id-remap fused. The build source for every engine above and the path for standalone onnxruntime use.

---

8. License & attribution

The encoder weights are intfloat/multilingual-e5-small (MIT), redistributed here in compiled form unchanged — MIT permits that redistribution and sublicensing, so this repository ships under Apache-2.0 with the encoder's MIT notice intact. The trained head and the build/quantization tooling are original. Backbone: XLM-RoBERTa. Not legal advice.

Used by the UltraCode code assistant, though nothing about the model is specific to it.