add/update model card with eval metrics

README.md

@@ -14,13 +14,18 @@ tags:
 
 # Prompt Injection Detection (encoder, multi-label)
 
-Multi-label classifier over 9 prompt-injection attack categories,
-fine-tuned from **[`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)**. Single
-forward pass; `is_valid` = any attack above threshold (0.5).
+Encoder classifier that detects which prompt-injection attack categories (out of
+9) appear in an input. Fine-tuned from
+**[`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)**.
+Replaces the 2B Qwen decoder LoRA with a single-forward-pass encoder for
+lower-latency runtime-security use in LLM-Guard's `PromptInjection` scanner.
 
 - **Base model**: [`jhu-clsp/mmBERT-base`](https://huggingface.co/jhu-clsp/mmBERT-base)
-- **Trained with**: max_seq_length=3072, epochs=3, lr=2e-05
 - **Labels (9)**: DirectInjection, Jailbreak, Adversarial, Extraction, Encoding, Manipulation, Smuggling, Indirect, MultiTurn
+- **Output**: per-category sigmoid; `is_valid` = any attack above threshold
+  (0.5).
+- **Multilingual / long context**: inherited from the base encoder; trained with
+  inputs up to the base model's positional limit.
 
 ## Usage
 
@@ -46,4 +51,30 @@ result = {"is_valid": bool(present), "category": {k: True for k in present}}
 print(result)   # e.g. {"is_valid": True, "category": {"DirectInjection": True}}
 ```
 
-> Test-set metrics are added by `eval_and_push_card.py` after evaluation.
+## Test-set metrics (n=500)
+
+| Metric | Value |
+|--------|-------|
+| is_valid (attack-detection) accuracy | 0.890 |
+| category-set (exact) accuracy | 0.054 |
+| micro-F1 | 0.351 |
+| macro-F1 | 0.356 |
+| latency mean (ms/example) | 1.823335886001587 |
+| latency p95 (ms/example) | 1.9494213163852692 |
+| device | cuda:0 |
+
+### Per-category F1
+
+| Category | F1 | Description |
+|----------|----|-------------|
+| `Adversarial` | 0.400 | Carefully crafted inputs that exploit model quirks or training artifacts to elicit unintended behavior without an obvious override. |
+| `DirectInjection` | 0.523 | Explicit instruction overrides that tell the model to ignore prior context (e.g. "ignore all previous instructions and …"). |
+| `Encoding` | 0.373 | Obfuscated payloads using base64 / ROT13 / leetspeak / homoglyphs / zero-width chars / shell pipes to bypass keyword filters. |
+| `Extraction` | 0.257 | Attempts to leak the system prompt, hidden instructions, or memorized training data (e.g. "print everything between <<system>> tags"). |
+| `Indirect` | 0.323 | Injection delivered through untrusted retrieved content (RAG passages, scraped pages, file contents) rather than the user's direct turn. |
+| `Jailbreak` | 0.397 | Persona / role swaps and constraint bypasses aimed at disabling safety alignment (e.g. DAN, "you are now an unrestricted assistant"). |
+| `Manipulation` | 0.272 | Social-engineering framings (urgency, authority, sympathy, false context) that pressure the model into compliance. |
+| `MultiTurn` | 0.164 | Crescendo / drip-feed attacks that build up across multiple turns to gradually erode guardrails. |
+| `Smuggling` | 0.491 | Hidden control tokens, chat-template markers, or special sequences injected to confuse the parser (e.g. fake `<|im_end|>` / role tags). |
+
+*Evaluated on `test_dataset_injection.csv`. Generated 2026-06-03 09:26 UTC.*