Add fine-tuned LaBSE checkpoint and model card

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,163 @@
+---
+language:
+- en
+- si
+- ta
+license: cc-by-4.0
+base_model: sentence-transformers/LaBSE
+pipeline_tag: text-classification
+tags:
+- banking
+- priority-classification
+- ticket-triage
+- labse
+- multilingual
+- code-mixed
+- sinhala
+- tamil
+metrics:
+- f1
+---
+
+# Swift-Support LaBSE Priority Classifier (v1.0)
+
+A fine-tuned **LaBSE** (Language-Agnostic BERT Sentence Embedding) model that assigns an
+escalation priority — **Low / Medium / High** — to a banking support ticket written in any of five
+language tracks. Built for the **Swift** support-ticket triage project, alongside
+[`Swift-Support/labse-intent-1.0`](https://huggingface.co/Swift-Support/labse-intent-1.0).
+
+## Model details
+
+* **Base architecture:** `sentence-transformers/LaBSE` (471M parameters, 501k vocabulary)
+* **Task:** 3-class text classification (priority / urgency)
+* **Classes:** `Low`, `Medium`, `High`
+* **Languages:** English, Sinhala, Tamil, Singlish (romanized Sinhala), Tanglish (romanized Tamil)
+* **Regime:** one multilingual model over all five tracks — *not* five per-language models
+
+The headline metric is **macro-F1**, never accuracy: the class distribution is roughly
+55% Low / 36% Medium / 9% High, so accuracy flatters a model that neglects `High`.
+
+## Evaluation
+
+Trained on `train+dev` (49,990 rows = 9,998 tickets × 5 languages), scored **once** on the
+held-out test set (15,395 rows = 3,079 tickets × 5 languages). Frozen split `e7b5934392cd`; test
+tickets come from the official BANKING77 test file and were never used for model selection.
+
+**Pooled test macro-F1: 0.8901** (accuracy 0.9008)
+
+Per-class F1: `Low` 0.9206 · `Medium` 0.8760 · `High` 0.8735
+
+### Against the alternatives (pooled test macro-F1)
+
+| model | macro-F1 |
+|---|---:|
+| `gemma-3-1b` multitask (shared head) | 0.8904 |
+| **LaBSE (this model)** | **0.8901** |
+| `gemma-3-1b` (LoRA, single-task) | 0.8898 |
+| mmBERT | 0.8887 |
+| XLM-RoBERTa base | 0.8872 |
+| TF-IDF + LinearSVC (classical champion) | 0.8722 |
+| TF-IDF + logistic regression | 0.8683 |
+
+The classical champion's 95% CI is [0.8605, 0.8831], so this model clears its upper bound — a
+real, if modest, win. The Gemma multitask variant is a statistical tie, not a better model.
+
+### Per language, on test
+
+| track | LaBSE (this model) | classical TF-IDF | delta |
+|---|---:|---:|---:|
+| English | **0.9229** | 0.9032 | +0.0197 |
+| Sinhala | **0.9179** | 0.8745 | +0.0434 |
+| Singlish (romanized) | 0.8817 | **0.8915** | −0.0098 |
+| Tamil | **0.9130** | 0.8905 | +0.0225 |
+| Tanglish (romanized) | **0.8142** | 0.7994 | +0.0148 |
+| **ALL (pooled)** | **0.8901** | 0.8722 | +0.0179 |
+
+Two things worth stating plainly:
+
+1. **The classical baseline still wins on Singlish.** LaBSE gives back most of its native-script
+   advantage on romanized text — a pattern that also shows up in linear probing, where LaBSE has
+   the *largest* native-minus-romanized gap of any backbone in the roster.
+2. **Tanglish is the weak track**, 7–10 points below every other language for both model families.
+
+## ⚠️ The label ceiling — read this before quoting 0.89
+
+The training labels were generated by an LLM prompt, not by human annotators. On a 500-ticket
+benchmark set that *was* annotated by hand, those prompt labels agree with human judgement at
+**0.7722 macro-F1** (95% CI [0.7263, 0.8147]; raw agreement 0.804, Cohen's κ = 0.644).
+
+This does **not** cap the number above — against the prompt labels a model could in principle
+reach 1.0. It caps what the number *means*. This model has learned the labeling rule well; the
+rule itself agrees with a human 77% of the time. **Quoting 0.89 as "priority accuracy" overstates
+what a human reviewer would call correct.** Any external write-up should state the 0.7722 figure
+alongside it.
+
+## Usage
+
+```python
+import torch
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+repo = "Swift-Support/labse-priority-1.0"
+tok = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
+
+texts = ["Someone has taken money from my account and nobody is helping me!",
+         "How do I activate my new card?"]
+
+with torch.no_grad():
+    batch = tok(texts, return_tensors="pt", padding=True, truncation=True, max_length=128)
+    probs = model(**batch).logits.softmax(-1)
+
+for text, p in zip(texts, probs):
+    print(model.config.id2label[int(p.argmax())], f"{p.max():.3f}", "|", text)
+```
+
+Two things that will silently corrupt results if you get them wrong:
+
+* **`max_length=128` must match training.** It is not stored in the checkpoint.
+* **Read the label from `model.config.id2label`, never from a hardcoded index.** This checkpoint
+  carries an explicit mapping (`0: Low, 1: Medium, 2: High`). A wrong index guess does not raise —
+  it silently returns the wrong priority.
+
+Cost: ~1.9 GB resident, roughly 100–300 ms per ticket on CPU. Load the model once at process
+start, never per request.
+
+## Training
+
+| | |
+|---|---|
+| base | `sentence-transformers/LaBSE` |
+| fit portion | `train+dev`, 49,990 rows |
+| epochs | 3 (best epoch: 3 of 3) |
+| learning rate | 2e-5 |
+| batch size | 32 |
+| max sequence length | 128 |
+| class imbalance | `class_weight` (balanced) |
+| precision | fp16 |
+| hardware | Kaggle T4, ~112 rows/s, 22 min wall |
+
+`best_epoch = 3 of 3` — the model was still improving when training stopped, which is what a
+consistent labeling target looks like.
+
+## Limitations
+
+* **Romanized text is synthetic.** Singlish is rule-generated from Sinhala and Tanglish is
+  machine-translated, so both are cleaner and more regular than text a human would type. The
+  Singlish and Tanglish numbers above are an optimistic upper bound, and no romanized-specific
+  conclusion from this model should be trusted until it is re-measured on human-typed data.
+* **Labels are LLM-generated** — see the label ceiling section.
+* **Domain-bound.** Derived from BANKING77; behaviour outside retail-banking support is untested.
+* **Not calibrated.** The softmax scores are not probabilities you should threshold on without
+  re-calibrating; a threshold tuned by cross-validation on a sibling task failed to transfer to
+  test in this project.
+* Trained and evaluated only on the five tracks listed. LaBSE covers 109 languages, but nothing
+  here measures the other 104.
+
+## Citation & provenance
+
+Derived from [BANKING77](https://huggingface.co/datasets/PolyAI/banking77) (PolyAI, CC-BY-4.0),
+translated into Sinhala and Tamil and romanized into Singlish and Tanglish. Priority labels were
+generated by an LLM prompt and benchmarked against human annotation as described above.
+
+Training data: [`Swift-Support/swift-support-tickets-1.0`](https://huggingface.co/datasets/Swift-Support/swift-support-tickets-1.0)

config.json

@@ -0,0 +1,46 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "directionality": "bidi",
+  "dtype": "float32",
+  "eos_token_id": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "Low",
+    "1": "Medium",
+    "2": "High"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "label2id": {
+    "Low": 0,
+    "Medium": 1,
+    "High": 2
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "pooler_fc_size": 768,
+  "pooler_num_attention_heads": 12,
+  "pooler_num_fc_layers": 3,
+  "pooler_size_per_head": 128,
+  "pooler_type": "first_token_transform",
+  "position_embedding_type": "absolute",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.0.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 501153
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03729b2bfd1373577ec344e3b45b41a82933bae2231e1ce66d1ceb2d05861157
+size 1883740532

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f72b993b05e566e04415720e03409ca251bf620ae9e003abe0aad1c08c10b6b
+size 13632017

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": false,
+  "full_tokenizer_file": null,
+  "is_local": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}