Feature Extraction
TensorRT
ONNX
OpenVINO
English
Russian
text-classification
denoising
multilingual
Instructions to use faxenoff/code-daemon-denoise-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use faxenoff/code-daemon-denoise-v1 with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -19,33 +19,21 @@ base_model:
|
|
| 19 |
|
| 20 |
A tiny, fast **bilingual (EN + RU) word denoiser** β it decides whether a single word form is a
|
| 21 |
**meaningful technical term** (keep) or **noise / ballast** (drop). It ships with the
|
| 22 |
-
[UltraCode](https://github.com/faxenoff/ultracode) MCP server, where it runs
|
| 23 |
-
(`SYS_BOOTSTRAP`) of the knowledge-graph pipeline: classifying the UNKNOWN word forms harvested from a
|
| 24 |
-
codebase's docs/identifiers so the search vocabulary stays clean.
|
| 25 |
-
|
| 26 |
-
It **replaces a prompt-based LLM denoiser** (Qwen2.5-Coder-1.5B answering YES/NO per word) with a
|
| 27 |
-
frozen encoder + a trained linear head. On the daemon's bootstrap pass this is **~60Γ faster**
|
| 28 |
-
(~0.8 s vs ~400 s) at the same quality, runs on the **CPU** (OpenVINO INT8), and never competes with
|
| 29 |
-
the main LLM for VRAM.
|
| 30 |
|
| 31 |
- **Frozen encoder** β [`intfloat/multilingual-e5-small`](https://huggingface.co/intfloat/multilingual-e5-small)
|
| 32 |
(XLM-RoBERTa, 384-dim), **no weight changes**. Mean-pooling + L2-norm are baked into the graph.
|
| 33 |
- **Trained linear head** β a logistic-regression probe (scikit-learn) over the 384-dim embedding,
|
| 34 |
**folded with its input scaler into a single affine** `P(keep) = sigmoid(wΒ·e + b)`. Ships as
|
| 35 |
-
`denoise_head.json` (`{dim, w[384], b, strip_threshold}`) β no Python at runtime; the daemon does
|
| 36 |
-
|
| 37 |
-
- **Vocab-pruned** β the 250k-token SentencePiece vocab is cut by character class to **Latin +
|
| 38 |
-
Cyrillic + punctuation (142k tokens)**, lossless for EN + RU, dropping the INT8 weights from ~121 MB
|
| 39 |
-
to **~76 MB**. The pruned-vocab id map is folded into a remap-Gather at the model input.
|
| 40 |
|
| 41 |
## How it was made
|
| 42 |
|
| 43 |
-
1. **Encoder**: export the frozen mE5-small to ONNX with mean-pool + L2-norm fused, prune the
|
| 44 |
-
embedding table to the kept character classes, and PTQ-quantize to INT8 (NNCF) for OpenVINO.
|
| 45 |
2. **Head**: embed a bilingual word-label set (EN: WordNet/BNC mid-frequency lemmas; RU:
|
| 46 |
Taiga/OpenCorpora/Nerus mid-Zipf) plus per-language manual gold, fit
|
| 47 |
-
`LogisticRegression(class_weight="balanced")`, then **fold** `StandardScaler` + LR into one
|
| 48 |
-
`(w, b)`. A `strip_threshold` (default **0.95**) trades strip precision vs recall.
|
| 49 |
|
| 50 |
Words are embedded with a fixed `"vocab: "` prefix (the daemon pads every candidate word the same
|
| 51 |
way) so very short inputs are not dropped by batch de-duplication β the head is trained on the
|
|
@@ -89,7 +77,8 @@ scores = p_keep(["mutex", "tensorrt", "ΠΏΠΎΠΆΠ°Π»ΡΠΉΡΡΠ°", "asdfgh"])
|
|
| 89 |
Pre-compiled, ready-to-run engines named per **runtime Γ GPU arch Γ OS** (single `s` bucket):
|
| 90 |
|
| 91 |
- **OpenVINO** `*_ov_cpu_int8_b64_s40.{xml,bin}` β Intel/AMD/any CPU, INT8 (the default lane).
|
| 92 |
-
- **TensorRT** `*_{win_x64,linux_x64}_trt_sm_{86,89,120}.engine` β NVIDIA,
|
|
|
|
| 93 |
- **TVM** `*_b64_s40_{win_x64,β¦}_tvm_vulkan.{dll,so}` β Vulkan fallback (optional GPU lane).
|
| 94 |
- **Head** β `denoise_head.json` (the trained affine; required).
|
| 95 |
- **Tokenizer** β `sentencepiece.bpe.model` (+ `tokenizer_config.json`). The daemon feeds raw
|
|
@@ -101,7 +90,7 @@ Pre-compiled, ready-to-run engines named per **runtime Γ GPU arch Γ OS** (sing
|
|
| 101 |
|
| 102 |
On a frozen held-out word set (EN + RU): **SAFE F1 β 0.79**, BALLAST F1 β 0.84, strip precision β
|
| 103 |
0.88 at `strip_threshold = 0.95`. The INT8 vocab-pruned build matches the full-vocab FP build (F1 0.79
|
| 104 |
-
vs 0.79) at 38 % of the size.
|
| 105 |
|
| 106 |
## License & attribution
|
| 107 |
|
|
|
|
| 19 |
|
| 20 |
A tiny, fast **bilingual (EN + RU) word denoiser** β it decides whether a single word form is a
|
| 21 |
**meaningful technical term** (keep) or **noise / ballast** (drop). It ships with the
|
| 22 |
+
[UltraCode](https://github.com/faxenoff/ultracode) MCP server, where it runs in the knowledge-graph pipeline: classifying the UNKNOWN word forms harvested from a codebase's docs/identifiers so the search vocabulary stays clean.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
- **Frozen encoder** β [`intfloat/multilingual-e5-small`](https://huggingface.co/intfloat/multilingual-e5-small)
|
| 25 |
(XLM-RoBERTa, 384-dim), **no weight changes**. Mean-pooling + L2-norm are baked into the graph.
|
| 26 |
- **Trained linear head** β a logistic-regression probe (scikit-learn) over the 384-dim embedding,
|
| 27 |
**folded with its input scaler into a single affine** `P(keep) = sigmoid(wΒ·e + b)`. Ships as
|
| 28 |
+
`denoise_head.json` (`{dim, w[384], b, strip_threshold}`) β no Python at runtime; the daemon does the dot product in-process.
|
| 29 |
+
- **Vocab-pruned** β the 250k-token SentencePiece vocab is cut by character class to **Latin + Cyrillic + punctuation (142k tokens)**, lossless for EN + RU, dropping the INT8 weights from ~121 MB to **~76 MB**. The pruned-vocab id map is folded into a remap-Gather at the model input.
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
## How it was made
|
| 32 |
|
| 33 |
+
1. **Encoder**: export the frozen mE5-small to ONNX with mean-pool + L2-norm fused, prune the embedding table to the kept character classes, and PTQ-quantize to INT8 (NNCF) for OpenVINO.
|
|
|
|
| 34 |
2. **Head**: embed a bilingual word-label set (EN: WordNet/BNC mid-frequency lemmas; RU:
|
| 35 |
Taiga/OpenCorpora/Nerus mid-Zipf) plus per-language manual gold, fit
|
| 36 |
+
`LogisticRegression(class_weight="balanced")`, then **fold** `StandardScaler` + LR into one `(w, b)`. A `strip_threshold` (default **0.95**) trades strip precision vs recall.
|
|
|
|
| 37 |
|
| 38 |
Words are embedded with a fixed `"vocab: "` prefix (the daemon pads every candidate word the same
|
| 39 |
way) so very short inputs are not dropped by batch de-duplication β the head is trained on the
|
|
|
|
| 77 |
Pre-compiled, ready-to-run engines named per **runtime Γ GPU arch Γ OS** (single `s` bucket):
|
| 78 |
|
| 79 |
- **OpenVINO** `*_ov_cpu_int8_b64_s40.{xml,bin}` β Intel/AMD/any CPU, INT8 (the default lane).
|
| 80 |
+
- **TensorRT** `*_{win_x64,linux_x64}_trt_sm_{86,89,120}.engine` β NVIDIA, BF16 (optional GPU lane;
|
| 81 |
+
the INT8 lane is OV CPU β this remap-baked SentencePiece ONNX isn't compatible with generic INT8 PTQ).
|
| 82 |
- **TVM** `*_b64_s40_{win_x64,β¦}_tvm_vulkan.{dll,so}` β Vulkan fallback (optional GPU lane).
|
| 83 |
- **Head** β `denoise_head.json` (the trained affine; required).
|
| 84 |
- **Tokenizer** β `sentencepiece.bpe.model` (+ `tokenizer_config.json`). The daemon feeds raw
|
|
|
|
| 90 |
|
| 91 |
On a frozen held-out word set (EN + RU): **SAFE F1 β 0.79**, BALLAST F1 β 0.84, strip precision β
|
| 92 |
0.88 at `strip_threshold = 0.95`. The INT8 vocab-pruned build matches the full-vocab FP build (F1 0.79
|
| 93 |
+
vs 0.79) at 38 % of the size.
|
| 94 |
|
| 95 |
## License & attribution
|
| 96 |
|