| --- |
| license: other |
| license_name: nutrient-commercial |
| library_name: transformers |
| pipeline_tag: text-classification |
| language: |
| - multilingual |
| - en |
| - de |
| - fr |
| - es |
| - zh |
| - ja |
| - ar |
| - hi |
| - ru |
| - tr |
| - vi |
| - ko |
| - sw |
| - ur |
| - th |
| tags: |
| - grounding |
| - hallucination-detection |
| - fact-verification |
| - nli |
| - zero-shot-classification |
| - multilingual |
| - document-ai |
| - cross-encoder |
| datasets: |
| - nutrientdocs/grounding-benchmark |
| metrics: |
| - roc_auc |
| --- |
| |
| # grounding-multilingual · _commercial_ |
|
|
| **Does the document actually support this claim — in 15+ languages?** `grounding-multilingual` is a |
| cross-encoder that scores whether a hypothesis (a number, date, or fact) is **entailed by** a premise |
| drawn from a real document — a |
| financial table, a filing, prose evidence. |
|
|
| It is the strongest, multilingual member of Nutrient's grounding model family. **Weights are commercial** |
| (not downloadable here); this page is a spec + scorecard. For the open, English model see |
| [`grounding-en`](https://huggingface.co/nutrientdocs/grounding-en). |
|
|
| - 🎯 **Try it:** [grounding-demo](https://huggingface.co/spaces/nutrientdocs/grounding-demo?model=multi) |
| - 🏆 **Leaderboard:** [grounding-leaderboard](https://huggingface.co/spaces/nutrientdocs/grounding-leaderboard) |
| - 📊 **Benchmark:** [grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark) |
|
|
| ## Results |
|
|
| On the held-out multilingual [grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark) |
| (ROC-AUC), against the top open multilingual NLI models: |
|
|
| | Facet | `grounding-multilingual` | mDeBERTa-v3-base XNLI-2mil7 | mDeBERTa-v3-base MNLI-XNLI | XLM-R-large XNLI | |
| | --- | ---: | ---: | ---: | ---: | |
| | **Overall** | **.965** | .894 | .831 | .811 | |
| | Number | **.999** | .775 | .806 | .765 | |
| | Table premises | **.999** | .727 | .780 | .715 | |
| | Prose premises | .926 | **.961** | .864 | .869 | |
|
|
| It leads on **number** (.999 vs .77–.81) and **table** grounding (.999 vs .72–.78) across 15+ languages; |
| on general prose NLI the best multilingual zero-shot model edges it. Full ranking on the |
| [leaderboard](https://huggingface.co/spaces/nutrientdocs/grounding-leaderboard). (Date/string grounding |
| is too rare in the multilingual corpus to score reliably, so it's omitted here.) |
|
|
| ## Calibrating the score |
|
|
| Fine-tuning maximizes _ranking_ (AUC), which can leave the raw probability overconfident. For a score you |
| can gate on ("0.9 means ~90% right"), apply **temperature scaling** — divide the logits by a fitted `T` |
| before softmax. It's monotonic, so AUC/ranking is untouched and only the confidence values are repaired. |
| On the serving distribution we fit **T = 0.94** (ECE 0.012 → 0.007). Re-fit `T` on your distribution |
| whenever your input pipeline changes. |
|
|
| ## Intended use & limits |
|
|
| - **Use it for:** verifying extracted values against source documents, hallucination/citation checking, |
| and routing low-confidence extractions for review — across 15+ languages, on-prem. |
| - **Limits:** the remaining ceiling is _reasoning_ table-claim negatives and multi-step |
| arithmetic. As a dedicated grounding checkpoint it trades a little general-NLI accuracy for grounding. |
|
|
| ## License & data |
|
|
| The model **weights** are offered under a commercial Nutrient license — on-prem, so documents never leave |
| your infrastructure. The training set is not redistributed. The **evaluation** data is public and |
| reproducible: [grounding-benchmark](https://huggingface.co/datasets/nutrientdocs/grounding-benchmark) |
| (CC-BY-SA-4.0); the full training set is not redistributed. |
|
|
| > ### 📩 Get access |
| > |
| > `grounding-multilingual` is commercial and its weights are not downloadable here. To run it on-prem — |
| > multilingual, calibrated, private — **contact Nutrient: [nutrient.io/contact-sales](https://www.nutrient.io/contact-sales/).** |
|
|
| ## About the author |
|
|
| <a href="https://nutrient.io/"> |
| <img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" /> |
| </a> |
|
|
| This project is maintained and funded by [Nutrient](https://nutrient.io/) - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks. |
