Berk commited on
Commit
8cb38ec
·
verified ·
1 Parent(s): dae602d

Add causal span model (mDeBERTa-v3 BIO tagger)

Browse files
Files changed (2) hide show
  1. README.md +12 -8
  2. pytorch_model.bin +2 -2
README.md CHANGED
@@ -35,17 +35,19 @@ on the [Causal News Corpus](https://github.com/tanfiona/CausalNewsCorpus) Subtas
35
  Official scorer (`evaluation/subtask2`: FairEval + best-combination alignment), V2 dev:
36
 
37
  ```
38
- Overall F1 0.696 (precision 0.697, recall 0.696)
39
- Cause F1 0.74 | Effect F1 0.69 | Signal F1 0.65
40
- Multi-relation sentences: F1 0.52 (beam top-2 decoding)
 
41
 
42
  Context (same official scorer):
43
- Organizer baseline (2023, dev) 0.627 <- this model beats it (+7 F1)
44
  1Cademy (2022 winner, test) 0.542
45
  BoschAI (2023 winner, test) 0.728
46
 
47
- Trained on English CNC Subtask-2 (relations=all), mDeBERTa-v3, lr 3e-5, 10 epochs.
48
- Augmented data was tried and HURT (0.675), so it is not used.
 
49
  ```
50
 
51
  **This beats the organizer's 0.627 dev baseline** and the 2022 shared-task winner
@@ -90,8 +92,10 @@ Verified on es/fr/de/pt/tr/ru/ar and CJK (zh/ja).
90
 
91
  - It is NOT compatible with a generic token-classification ONNX consumer -- it
92
  needs its own start/end + beam-search decoder (provided by the package).
93
- - It is a span EXTRACTOR that assumes the input is causal (no causal/non-causal
94
- gate), so gate it before feeding arbitrary text.
 
 
95
 
96
  ## License
97
 
 
35
  Official scorer (`evaluation/subtask2`: FairEval + best-combination alignment), V2 dev:
36
 
37
  ```
38
+ Overall F1 0.689 (span extraction; with the causal gate + beam dedup)
39
+ Cause F1 0.72 | Effect F1 0.69 | Signal F1 0.65
40
+ Multi-relation sentences: F1 0.50 (beam top-2 decoding)
41
+ Causal gate: accuracy 0.85 (precision 0.86, recall 0.87) on CNC dev
42
 
43
  Context (same official scorer):
44
+ Organizer baseline (2023, dev) 0.627 <- this model beats it
45
  1Cademy (2022 winner, test) 0.542
46
  BoschAI (2023 winner, test) 0.728
47
 
48
+ Trained on English CNC Subtask-2 (relations=all) + 1451 non-causal negatives for the
49
+ gate, mDeBERTa-v3, lr 3e-5, 10 epochs. Multilingual at inference (script-aware
50
+ segmentation). Augmented data was tried and hurt, so it is unused.
51
  ```
52
 
53
  **This beats the organizer's 0.627 dev baseline** and the 2022 shared-task winner
 
92
 
93
  - It is NOT compatible with a generic token-classification ONNX consumer -- it
94
  needs its own start/end + beam-search decoder (provided by the package).
95
+ - It has a built-in **causal gate** (a causal/non-causal head, ~0.85 accuracy on
96
+ CNC dev): `predict_relations` returns `[]` on text it judges non-causal, so it
97
+ is safe to run on arbitrary input. Beam duplicates are collapsed to one relation
98
+ per distinct cause->effect.
99
 
100
  ## License
101
 
pytorch_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7c9b5d9411e0567e77ac9f172691112e820b87a59a46599f3c8038f3971ffec1
3
- size 1112985739
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d2b0a343112a5f8d906ace2ec22392aa6104e319ec5dbb28e68e842241fd408
3
+ size 1112992531