Instructions to use SlayerLab/NERGAL with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use SlayerLab/NERGAL with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="SlayerLab/NERGAL")# Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("SlayerLab/NERGAL") model = AutoModelForTokenClassification.from_pretrained("SlayerLab/NERGAL", device_map="auto") - Notebooks
- Google Colab
- Kaggle
limits
Browse files
README.md
CHANGED
|
@@ -28,6 +28,43 @@ Python rules do the identifiers they can prove. A transformer NER head adds phon
|
|
| 28 |
- **Additive labels:** XLM-RoBERTa-large token classifier, BIO tags `phone` / `pii`, threshold 0.95
|
| 29 |
- **This snapshot:** seed `202609160`, **epoch 5** of a seven-epoch schedule
|
| 30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
## Versions
|
| 32 |
|
| 33 |
841-dev, union at 0.95, 354 gold spans. Same weights and API is a patch; new capability is minor; API, threshold, or weight recipe is major. A release that changes these numbers updates `hybrid.json` `eval` and this table.
|
|
|
|
| 28 |
- **Additive labels:** XLM-RoBERTa-large token classifier, BIO tags `phone` / `pii`, threshold 0.95
|
| 29 |
- **This snapshot:** seed `202609160`, **epoch 5** of a seven-epoch schedule
|
| 30 |
|
| 31 |
+
## What NERGAL detects — and what it does not
|
| 32 |
+
|
| 33 |
+
NERGAL masks **contact details and selected identifiers in Polish text**. It is not a general-purpose anonymizer: names, postal addresses and other personal information can remain in the output. Its corpus-masking policy also includes public, institutional and company contacts and identifiers.
|
| 34 |
+
|
| 35 |
+
### Detection scope
|
| 36 |
+
|
| 37 |
+
These are target categories, not a guarantee that every occurrence or format is detected.
|
| 38 |
+
|
| 39 |
+
| Category | Values in scope | Replacement |
|
| 40 |
+
|---|---|---|
|
| 41 |
+
| Phone contacts | Phone, fax and SMS contact numbers, including foreign, emergency, short/service and vanity numbers; extensions and number alternatives | `[Telefon]` |
|
| 42 |
+
| Email | Email addresses, including recoverable broken or incomplete addresses | `[PII]` |
|
| 43 |
+
| Personal identifiers | PESEL, passport and identity-document numbers | `[PII]` |
|
| 44 |
+
| Organization identifiers | NIP/VAT, REGON, KRS, GEMI, LEI and equivalent foreign company registration numbers; labelled DUNS, BDO and RPWDL register-book numbers | `[PII]` |
|
| 45 |
+
| Financial identifiers | Bank/account numbers, including Polish accounts and foreign IBANs | `[PII]` |
|
| 46 |
+
| Property identifiers | Land-register (księga wieczysta, KW) numbers | `[PII]` |
|
| 47 |
+
| Electronic contacts and access | e-Doręczenia and ePUAP addresses, explicitly labelled numeric access PINs (including `pin=` URL values), GG account IDs | `[PII]` |
|
| 48 |
+
|
| 49 |
+
Context matters: a number that resembles a phone or identifier is not automatically in scope. Rules use labels, format checks and, for some unlabelled identifiers, checksums; the model adds contextual detections. Coverage varies by category, and the aggregate benchmark below does not establish recall for every category.
|
| 50 |
+
|
| 51 |
+
### Outside detection scope
|
| 52 |
+
|
| 53 |
+
NERGAL is not designed to remove:
|
| 54 |
+
|
| 55 |
+
- **Personal names**, including private individuals and public officials; organization names.
|
| 56 |
+
- **Postal/street addresses, dates of birth and ages.**
|
| 57 |
+
- **Social-media handles, ordinary URLs and filenames.** An in-scope value inside a URL, such as a labelled numeric PIN, can still be masked.
|
| 58 |
+
- **Vehicle registration plates and generic serial, model or version codes.**
|
| 59 |
+
- **Document, case, article, funding and procurement references**, including procedure UUIDs; ISBNs, ORCIDs, TERYT codes, EAN/GTIN product codes and CNIL website-registration references.
|
| 60 |
+
- Prices, list numbers, generic labels without values, clearly fictitious examples and already-redacted placeholders.
|
| 61 |
+
|
| 62 |
+
These are intended exclusions; false positives can still mask some of this content.
|
| 63 |
+
|
| 64 |
+
### Known gaps in 1.0.3
|
| 65 |
+
|
| 66 |
+
Unlabelled phones and identifiers, unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 1.0.3 model. Do not rely on it to remove them consistently.
|
| 67 |
+
|
| 68 |
## Versions
|
| 69 |
|
| 70 |
841-dev, union at 0.95, 354 gold spans. Same weights and API is a patch; new capability is minor; API, threshold, or weight recipe is major. A release that changes these numbers updates `hybrid.json` `eval` and this table.
|