ppuzio commited on
Commit
3f67c66
·
1 Parent(s): 2bf5c25
Files changed (1) hide show
  1. README.md +37 -0
README.md CHANGED
@@ -28,6 +28,43 @@ Python rules do the identifiers they can prove. A transformer NER head adds phon
28
  - **Additive labels:** XLM-RoBERTa-large token classifier, BIO tags `phone` / `pii`, threshold 0.95
29
  - **This snapshot:** seed `202609160`, **epoch 5** of a seven-epoch schedule
30
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  ## Versions
32
 
33
  841-dev, union at 0.95, 354 gold spans. Same weights and API is a patch; new capability is minor; API, threshold, or weight recipe is major. A release that changes these numbers updates `hybrid.json` `eval` and this table.
 
28
  - **Additive labels:** XLM-RoBERTa-large token classifier, BIO tags `phone` / `pii`, threshold 0.95
29
  - **This snapshot:** seed `202609160`, **epoch 5** of a seven-epoch schedule
30
 
31
+ ## What NERGAL detects — and what it does not
32
+
33
+ NERGAL masks **contact details and selected identifiers in Polish text**. It is not a general-purpose anonymizer: names, postal addresses and other personal information can remain in the output. Its corpus-masking policy also includes public, institutional and company contacts and identifiers.
34
+
35
+ ### Detection scope
36
+
37
+ These are target categories, not a guarantee that every occurrence or format is detected.
38
+
39
+ | Category | Values in scope | Replacement |
40
+ |---|---|---|
41
+ | Phone contacts | Phone, fax and SMS contact numbers, including foreign, emergency, short/service and vanity numbers; extensions and number alternatives | `[Telefon]` |
42
+ | Email | Email addresses, including recoverable broken or incomplete addresses | `[PII]` |
43
+ | Personal identifiers | PESEL, passport and identity-document numbers | `[PII]` |
44
+ | Organization identifiers | NIP/VAT, REGON, KRS, GEMI, LEI and equivalent foreign company registration numbers; labelled DUNS, BDO and RPWDL register-book numbers | `[PII]` |
45
+ | Financial identifiers | Bank/account numbers, including Polish accounts and foreign IBANs | `[PII]` |
46
+ | Property identifiers | Land-register (księga wieczysta, KW) numbers | `[PII]` |
47
+ | Electronic contacts and access | e-Doręczenia and ePUAP addresses, explicitly labelled numeric access PINs (including `pin=` URL values), GG account IDs | `[PII]` |
48
+
49
+ Context matters: a number that resembles a phone or identifier is not automatically in scope. Rules use labels, format checks and, for some unlabelled identifiers, checksums; the model adds contextual detections. Coverage varies by category, and the aggregate benchmark below does not establish recall for every category.
50
+
51
+ ### Outside detection scope
52
+
53
+ NERGAL is not designed to remove:
54
+
55
+ - **Personal names**, including private individuals and public officials; organization names.
56
+ - **Postal/street addresses, dates of birth and ages.**
57
+ - **Social-media handles, ordinary URLs and filenames.** An in-scope value inside a URL, such as a labelled numeric PIN, can still be masked.
58
+ - **Vehicle registration plates and generic serial, model or version codes.**
59
+ - **Document, case, article, funding and procurement references**, including procedure UUIDs; ISBNs, ORCIDs, TERYT codes, EAN/GTIN product codes and CNIL website-registration references.
60
+ - Prices, list numbers, generic labels without values, clearly fictitious examples and already-redacted placeholders.
61
+
62
+ These are intended exclusions; false positives can still mask some of this content.
63
+
64
+ ### Known gaps in 1.0.3
65
+
66
+ Unlabelled phones and identifiers, unusual formatting and damaged text can escape detection. **VINs and obfuscated emails** (such as `name (at) domain.pl`) are approved annotation targets, but that approval alone does not establish reliable support in the released 1.0.3 model. Do not rely on it to remove them consistently.
67
+
68
  ## Versions
69
 
70
  841-dev, union at 0.95, 354 gold spans. Same weights and API is a patch; new capability is minor; API, threshold, or weight recipe is major. A release that changes these numbers updates `hybrid.json` `eval` and this table.