Create README.md

README.md

@@ -0,0 +1,174 @@
+---
+language:
+- de
+- en
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- retrieval
+- german
+- mteb
+base_model: nvidia/Nemotron-3-Embed-1B-BF16
+license: cc-by-nc-4.0
+---
+
+# most-embed-de: A German Retrieval-Optimized Embedding Model
+
+![most-embed-de-hero-banner-medium](https://cdn-uploads.huggingface.co/production/uploads/5efda656ff69163f6f59e5d2/b6j5lk8-PlLQTehjo65Nn.png)
+
+**most-embed-de** is a 1.1B-parameter German text-embedding model optimized for the **retrieval in German customer-support applications**. It acts as the
+retriever behind a support RAG system, an FAQ/help-center search, or an agent that has to find the one
+passage in a company's own documentation that resolves a customer's request.
+
+It is a fine-tune of **[nvidia/Nemotron-3-Embed-1B-BF16](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16)**
+(2048-dim embeddings, up to 32k context). Among models it can be deployed next to, it is:
+
+- **#1 in its size class (≤1.5B) on the German retrieval + reranking cut of MTEB(deu)** — and it beats
+  the 7.6B `Qwen3-Embedding-8B` and the 14B `F2LLM-v2-14B` while being **~7–12× smaller**.
+- **#1 in its size class on customer-support retrieval** (our SupportIR-DE benchmark), second only to the Nemotron-3-Embed
+  7.6B model and again ahead of `Qwen3-Embedding-8B` — at a fraction of the serving cost.
+- Target language: **German**
+- Base model languages: English, Arabic, Assamese, Bengali, Bulgarian, Chinese, Danish, Dutch, Finnish, French, Hindi, Hinglish, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Nepali, Norwegian, Persian, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Tamil, Telugu, Thai, Ukrainian, Urdu, Vietnamese.
+
+At 1.1B parameters it runs comfortably on a single commodity GPU, so you get 7–14B-class German support
+retrieval quality at ~1B-class latency and cost.
+
+## Why this model exists
+
+Public German retrieval benchmarks like the German slice of MTEB and RTEB are dominated by
+**Wikipedia-style open-domain QA and legal question answering**. They are excellent academic tasks, but
+they are **not** what many real world applications such a customer-support RAG systems need. A support retriever has to:
+
+- work **within a single company's index** (a query about *your* return policy must not match a
+  *competitor's* return policy), and
+- handle the real register of support queries — terse keyword searches, full questions, and statement-style
+  descriptions of a problem — across very different **industries** (telco, banking, insurance, e-commerce),
+  each with its own vocabulary and document conventions.
+
+None of the public German benchmarks measure this. So we built one.
+
+
+## Evaluation
+
+### German Retrieval
+
+![german-retrieval](https://cdn-uploads.huggingface.co/production/uploads/5efda656ff69163f6f59e5d2/ERv5HapZ5fPOjm65nnclD.png)
+
+German Retrieval is essentially the [**Retrieval + Reranking subset of MTEB(deu)**](https://mteb-leaderboard.hf.space/) — LegalQuAD,
+GerDaLIR (small, for efficiency), GermanDPR, GermanQuAD, XMarket, and MIRACL(de) — scored with the
+official MTEB methodology (mean of nDCG@10, MRR@5 for GermanQuAD). Baselines are the published leaderboard
+numbers; most-embed-de is measured on the identical task suite.
+
+### SupportIR-DE — customer-support retrieval
+
+![supportir-de](https://cdn-uploads.huggingface.co/production/uploads/5efda656ff69163f6f59e5d2/yaGrfFK3HksVny9incG3p.png)
+
+SupportIR-DE scores **company-scoped nDCG@10**: every query is ranked only against *its own company's*
+documents — the index a real support RAG system actually searches — across four industries, **held out by
+company in 5-fold cross-validation** (mean ± std over folds).
+
+Because no public German benchmark reflects support retrieval, we designed **SupportIR-DE** as part of
+this project:
+
+- **Four industries** — telecommunications, retail banking & payments, insurance, and e-commerce /
+  omni-channel retail — each with real German help-center content from public Web pages and sourced from the [Common Crawl corpus](https://commoncrawl.org).
+- **Company-scoped task** — every query is evaluated against only its own company's document index
+  (brand/company disambiguation is part of the task, exactly as in deployment). A pooled-industry variant
+  is reported as a diagnostic.
+- **Multiple query registers** — natural questions, keyword searches, and statement-style problem
+  descriptions, so a model cannot look strong on one register alone.
+- **Held out by company** (k-fold cross-validation over companies) with confidence intervals, plus a
+  human-authored FAQ gold slice for judge calibration.
+
+SupportIR-DE is **not yet public**. 
+
+
+## How it was built — autonomous, agent-driven development
+
+
+![agent-model-building-lifecycle-medium](https://cdn-uploads.huggingface.co/production/uploads/5efda656ff69163f6f59e5d2/KzvsH_78UOH9ZtAY9TV5e.png)
+
+The distinguishing feature of this project is not a single training trick; it is the **development
+process**. most-embed-de was produced by an **autonomous, agent-driven research pipeline** that carried the
+work through the *entire* model-building lifecycle:
+
+- **literature and landscape research** (what German retrieval benchmarks exist, where they fall short),
+- **benchmark selection and design** (choosing the public evaluation cut; designing SupportIR-DE),
+- **data curation** (sourcing and cleaning German support corpora; generating and quality-judging training
+  data; contamination and leakage guards),
+- **experimental design** (choosing the base model, the training and retention mix, and the ablations),
+- **scheduling and operating training + evaluation jobs on a GPU cluster**, and
+- **evaluation and error analysis** (including catching a train/test leakage issue in a retention pool and
+  correcting the reported numbers).
+
+The approach was inspired by [Andrej Karpathy's autoresearch project](https://github.com/karpathy/autoresearch). A human
+directed the objectives and reviewed every decision gate; the agents did the research, engineering, and
+analysis in between. The result is a stat
+
+## Usage
+
+The model can be used with all frameworks and inference systems that support Nemotron-3-Embed:
+
+### Sentence Transformers
+
+```python
+import torch
+from sentence_transformers import SentenceTransformer
+
+MODEL_ID = "malteos/most-embed-de"
+
+model = SentenceTransformer(
+    MODEL_ID,
+    device="cuda",
+    model_kwargs={
+        "dtype": torch.bfloat16,
+        "attn_implementation": "flash_attention_2",  # optional
+    },
+)
+model.max_seq_length = 32768
+
+QUERIES = [
+    "Write a Python function that counts the frequency of each element in a list of lists.",
+    "Write a function that orders a dictionary with tuple keys by the product of each key's tuple values.",
+    "What symptoms and common triggers help distinguish eczema from other inflammatory skin conditions?",
+    "How can someone reduce exposure to pollen during allergy season?",
+]
+
+DOCUMENTS = [
+    "def frequency_lists(list1):\n    flattened = [item for sublist in list1 for item in sublist]\n    counts = {}\n    for item in flattened:\n        if item in counts:\n            counts[item] += 1\n        else:\n            counts[item] = 1\n    return counts",
+    "def sort_dict_item(test_dict):\n    return {key: test_dict[key] for key in sorted(test_dict.keys(), key=lambda ele: ele[0] * ele[1])}",
+    "Eczema commonly causes itchy, dry, inflamed patches of skin. The affected areas may look red, scaly, cracked, or darker than the surrounding skin depending on skin tone. Symptoms can flare after exposure to irritants, allergens, stress, or changes in weather.",
+    "People with pollen allergy can reduce exposure by staying indoors on dry, windy days, avoiding early-morning outdoor activity, and going outside after rain when pollen levels are lower. They should check pollen forecasts, close windows and doors when counts are high, and consider starting allergy medication before symptoms begin if high pollen is expected. After being outside, showering, changing clothes, avoiding outdoor laundry drying, and wearing a face mask for yard work can help limit pollen contact.",
+]
+query_embeddings = model.encode_query(QUERIES, batch_size=8, convert_to_tensor=True)
+document_embeddings = model.encode_document(DOCUMENTS, batch_size=8, convert_to_tensor=True)
+
+scores = model.similarity(query_embeddings, document_embeddings)
+print("Similarity scores:")
+print(f"{'':>4}" + "".join(f"d[{i}]".rjust(10) for i in range(scores.shape[1])))
+for query_index, row in enumerate(scores):
+    print(f"q[{query_index}]" + "".join(f"{score.item():>10.4f}" for score in row))
+```
+
+### vLLM Online Serving
+
+```bash
+MODEL_ID=malteos/most-embed-de
+
+vllm serve "$MODEL_ID"
+```
+
+vLLM defaults to port 8000. Add host and port when you need an explicit bind address or a non-default port:
+
+```bash
+vllm serve "$MODEL_ID" --host 0.0.0.0 --port 8000
+```
+
+For other frameworks, see the [usage section in the Nemotron model card](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16#usage).
+
+
+## License
+
+**CC-BY-NC-4.0** — free for research and non-commercial use with attribution. [Reach out for commercial use or extended support](https://ostendorff.org/contact/).