Add causal span model (mDeBERTa-v3 BIO tagger)
Browse files- README.md +12 -8
- pytorch_model.bin +2 -2
README.md
CHANGED
|
@@ -35,17 +35,19 @@ on the [Causal News Corpus](https://github.com/tanfiona/CausalNewsCorpus) Subtas
|
|
| 35 |
Official scorer (`evaluation/subtask2`: FairEval + best-combination alignment), V2 dev:
|
| 36 |
|
| 37 |
```
|
| 38 |
-
Overall F1 0.
|
| 39 |
-
Cause F1 0.
|
| 40 |
-
Multi-relation sentences: F1 0.
|
|
|
|
| 41 |
|
| 42 |
Context (same official scorer):
|
| 43 |
-
Organizer baseline (2023, dev) 0.627 <- this model beats it
|
| 44 |
1Cademy (2022 winner, test) 0.542
|
| 45 |
BoschAI (2023 winner, test) 0.728
|
| 46 |
|
| 47 |
-
Trained on English CNC Subtask-2 (relations=all)
|
| 48 |
-
|
|
|
|
| 49 |
```
|
| 50 |
|
| 51 |
**This beats the organizer's 0.627 dev baseline** and the 2022 shared-task winner
|
|
@@ -90,8 +92,10 @@ Verified on es/fr/de/pt/tr/ru/ar and CJK (zh/ja).
|
|
| 90 |
|
| 91 |
- It is NOT compatible with a generic token-classification ONNX consumer -- it
|
| 92 |
needs its own start/end + beam-search decoder (provided by the package).
|
| 93 |
-
- It
|
| 94 |
-
|
|
|
|
|
|
|
| 95 |
|
| 96 |
## License
|
| 97 |
|
|
|
|
| 35 |
Official scorer (`evaluation/subtask2`: FairEval + best-combination alignment), V2 dev:
|
| 36 |
|
| 37 |
```
|
| 38 |
+
Overall F1 0.689 (span extraction; with the causal gate + beam dedup)
|
| 39 |
+
Cause F1 0.72 | Effect F1 0.69 | Signal F1 0.65
|
| 40 |
+
Multi-relation sentences: F1 0.50 (beam top-2 decoding)
|
| 41 |
+
Causal gate: accuracy 0.85 (precision 0.86, recall 0.87) on CNC dev
|
| 42 |
|
| 43 |
Context (same official scorer):
|
| 44 |
+
Organizer baseline (2023, dev) 0.627 <- this model beats it
|
| 45 |
1Cademy (2022 winner, test) 0.542
|
| 46 |
BoschAI (2023 winner, test) 0.728
|
| 47 |
|
| 48 |
+
Trained on English CNC Subtask-2 (relations=all) + 1451 non-causal negatives for the
|
| 49 |
+
gate, mDeBERTa-v3, lr 3e-5, 10 epochs. Multilingual at inference (script-aware
|
| 50 |
+
segmentation). Augmented data was tried and hurt, so it is unused.
|
| 51 |
```
|
| 52 |
|
| 53 |
**This beats the organizer's 0.627 dev baseline** and the 2022 shared-task winner
|
|
|
|
| 92 |
|
| 93 |
- It is NOT compatible with a generic token-classification ONNX consumer -- it
|
| 94 |
needs its own start/end + beam-search decoder (provided by the package).
|
| 95 |
+
- It has a built-in **causal gate** (a causal/non-causal head, ~0.85 accuracy on
|
| 96 |
+
CNC dev): `predict_relations` returns `[]` on text it judges non-causal, so it
|
| 97 |
+
is safe to run on arbitrary input. Beam duplicates are collapsed to one relation
|
| 98 |
+
per distinct cause->effect.
|
| 99 |
|
| 100 |
## License
|
| 101 |
|
pytorch_model.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d2b0a343112a5f8d906ace2ec22392aa6104e319ec5dbb28e68e842241fd408
|
| 3 |
+
size 1112992531
|