Text Classification
Safetensors
PyTorch
English
phishbyte
phishing-detection
email-security
cybersecurity
security
from-scratch
no-pretrained-weights
cascading-inference
lightweight
explainable-ai
nlp
phishing
spam-detection
malware-detection
threat-detection
email-classification
feature-engineering
interpretable-ml
Eval Results (legacy)
Expand tags, add widget examples, add metrics block
Browse files
README.md
CHANGED
|
@@ -1,19 +1,31 @@
|
|
| 1 |
---
|
| 2 |
-
language:
|
|
|
|
| 3 |
license: mit
|
| 4 |
library_name: phishbyte
|
| 5 |
pipeline_tag: text-classification
|
| 6 |
tags:
|
| 7 |
- phishing-detection
|
| 8 |
- email-security
|
|
|
|
|
|
|
| 9 |
- pytorch
|
| 10 |
- from-scratch
|
| 11 |
- no-pretrained-weights
|
| 12 |
- cascading-inference
|
| 13 |
- lightweight
|
| 14 |
- explainable-ai
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
datasets:
|
| 16 |
-
-
|
| 17 |
metrics:
|
| 18 |
- f1
|
| 19 |
- precision
|
|
@@ -31,12 +43,21 @@ model-index:
|
|
| 31 |
metrics:
|
| 32 |
- type: f1
|
| 33 |
value: 0.948
|
|
|
|
| 34 |
- type: accuracy
|
| 35 |
value: 0.944
|
|
|
|
| 36 |
- type: precision
|
| 37 |
-
value: 0.
|
|
|
|
| 38 |
- type: recall
|
| 39 |
-
value: 0.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
---
|
| 41 |
|
| 42 |
# Phish_Byte
|
|
@@ -45,83 +66,94 @@ A from-scratch PyTorch model for **email phishing detection**.
|
|
| 45 |
**F1 0.948** on CEAS-2008. **12,545 parameters** (β9,000Γ smaller than DistilBERT).
|
| 46 |
**1,500+ emails/sec** on a laptop GPU. Every verdict explains itself.
|
| 47 |
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
Every phishing detection model on HuggingFace is currently a fine-tuned
|
| 51 |
-
transformer (DistilBERT, BERT, RoBERTa) β 65 to 110 million parameters,
|
| 52 |
-
~250 MB on disk, ~50 ms per email on GPU. Phish_Byte takes a different
|
| 53 |
-
bet: a small custom MLP trained from scratch, fed by 29 carefully chosen
|
| 54 |
-
features, routed through a cascading inference pipeline. The model is
|
| 55 |
-
**9,000Γ smaller** than DistilBERT, performs competitively, deploys
|
| 56 |
-
without a GPU, and explains every decision.
|
| 57 |
|
| 58 |
-
##
|
| 59 |
|
| 60 |
```python
|
| 61 |
from phishbyte import PhishByteEngine
|
| 62 |
|
| 63 |
-
engine = PhishByteEngine.from_pretrained("
|
| 64 |
verdict = engine.analyze(raw_email_string)
|
| 65 |
|
| 66 |
print(verdict.label) # 'phishing'
|
| 67 |
print(verdict.probability) # 0.9735
|
| 68 |
print(verdict.confidence) # 'high'
|
| 69 |
-
print(verdict.layer_used) # 2
|
| 70 |
-
print(verdict.feature_weights) #
|
| 71 |
```
|
| 72 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 73 |
## Architecture
|
| 74 |
|
| 75 |
```
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
ββββΊ otherwise route to MLP
|
| 81 |
-
β
|
| 82 |
-
Layer 2 β MLP (~3 ms): 29 β 96 β 48 β 1 (sigmoid)
|
| 83 |
-
β
|
| 84 |
-
βΌ
|
| 85 |
-
PhishVerdict {label, probability, confidence, layer_used, feature_weights}
|
| 86 |
```
|
| 87 |
|
| 88 |
-
##
|
| 89 |
-
|
| 90 |
-
| Metric | Value |
|
| 91 |
-
|------------------|----------:|
|
| 92 |
-
| F1 score | **0.948** |
|
| 93 |
-
| Accuracy | 94.40% |
|
| 94 |
-
| Precision | 0.9537 |
|
| 95 |
-
| Recall | 0.9432 |
|
| 96 |
-
| Parameters | 12,545 |
|
| 97 |
-
| Model size | ~50 KB |
|
| 98 |
-
| Throughput (GPU) | 1,527 /s |
|
| 99 |
-
| Throughput (CPU) | ~800 /s |
|
| 100 |
|
| 101 |
-
|
|
|
|
|
|
|
| 102 |
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
- **Composite (4)**: per-layer normalized scores
|
| 109 |
|
| 110 |
## Limitations
|
| 111 |
|
| 112 |
-
- ~5%
|
| 113 |
-
- Trained on CEAS-2008
|
| 114 |
-
- SPF validation
|
| 115 |
-
- Adversarial emails crafted specifically to game these features will get through.
|
| 116 |
|
| 117 |
## Citation
|
| 118 |
|
| 119 |
```bibtex
|
| 120 |
@software{phishbyte2026,
|
| 121 |
-
author
|
| 122 |
-
title
|
| 123 |
-
year
|
| 124 |
-
url
|
| 125 |
}
|
| 126 |
```
|
| 127 |
|
|
|
|
| 1 |
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
license: mit
|
| 5 |
library_name: phishbyte
|
| 6 |
pipeline_tag: text-classification
|
| 7 |
tags:
|
| 8 |
- phishing-detection
|
| 9 |
- email-security
|
| 10 |
+
- cybersecurity
|
| 11 |
+
- security
|
| 12 |
- pytorch
|
| 13 |
- from-scratch
|
| 14 |
- no-pretrained-weights
|
| 15 |
- cascading-inference
|
| 16 |
- lightweight
|
| 17 |
- explainable-ai
|
| 18 |
+
- nlp
|
| 19 |
+
- phishing
|
| 20 |
+
- spam-detection
|
| 21 |
+
- malware-detection
|
| 22 |
+
- threat-detection
|
| 23 |
+
- email-classification
|
| 24 |
+
- text-classification
|
| 25 |
+
- feature-engineering
|
| 26 |
+
- interpretable-ml
|
| 27 |
datasets:
|
| 28 |
+
- ceas-2008
|
| 29 |
metrics:
|
| 30 |
- f1
|
| 31 |
- precision
|
|
|
|
| 43 |
metrics:
|
| 44 |
- type: f1
|
| 45 |
value: 0.948
|
| 46 |
+
name: F1 Score
|
| 47 |
- type: accuracy
|
| 48 |
value: 0.944
|
| 49 |
+
name: Accuracy
|
| 50 |
- type: precision
|
| 51 |
+
value: 0.9537
|
| 52 |
+
name: Precision
|
| 53 |
- type: recall
|
| 54 |
+
value: 0.9432
|
| 55 |
+
name: Recall
|
| 56 |
+
widget:
|
| 57 |
+
- text: "From: PayPal Security <security@paypa1-alert.tk>\nReply-To: attacker@evil-domain.ru\nSubject: URGENT: Your account will be suspended\n\nDear Customer, your PayPal account has been suspended. Verify now at http://paypal-login.tk/verify"
|
| 58 |
+
example_title: "Phishing email example"
|
| 59 |
+
- text: "From: alice@company.com\nReply-To: alice@company.com\nSubject: Team lunch tomorrow\n\nHi everyone, lunch is at noon tomorrow in the usual spot. See you there!"
|
| 60 |
+
example_title: "Legitimate email example"
|
| 61 |
---
|
| 62 |
|
| 63 |
# Phish_Byte
|
|
|
|
| 66 |
**F1 0.948** on CEAS-2008. **12,545 parameters** (β9,000Γ smaller than DistilBERT).
|
| 67 |
**1,500+ emails/sec** on a laptop GPU. Every verdict explains itself.
|
| 68 |
|
| 69 |
+
> **v3 in progress:** expanding to 50K parameters + 6-dataset corpus training.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
+
## Quick start
|
| 72 |
|
| 73 |
```python
|
| 74 |
from phishbyte import PhishByteEngine
|
| 75 |
|
| 76 |
+
engine = PhishByteEngine.from_pretrained("SamSec007/phishbyte")
|
| 77 |
verdict = engine.analyze(raw_email_string)
|
| 78 |
|
| 79 |
print(verdict.label) # 'phishing'
|
| 80 |
print(verdict.probability) # 0.9735
|
| 81 |
print(verdict.confidence) # 'high'
|
| 82 |
+
print(verdict.layer_used) # 2
|
| 83 |
+
print(verdict.feature_weights) # per-feature attribution
|
| 84 |
```
|
| 85 |
|
| 86 |
+
## Why this exists
|
| 87 |
+
|
| 88 |
+
Every phishing detection model on HuggingFace is a fine-tuned transformer β
|
| 89 |
+
DistilBERT, BERT, RoBERTa. 65β110M parameters. ~250 MB on disk. ~50 ms/email.
|
| 90 |
+
|
| 91 |
+
Phish_Byte is different:
|
| 92 |
+
- Custom MLP trained **from scratch** β no pretrained weights
|
| 93 |
+
- **29 engineered features** across domain, URL, SPF, subject, and character-level signals
|
| 94 |
+
- **Cascading inference** β cheap rules handle obvious cases, MLP handles the rest
|
| 95 |
+
- **Full email header analysis** including live SPF validation
|
| 96 |
+
- Runs on **CPU without a GPU**
|
| 97 |
+
- Every verdict includes **which signals fired and why**
|
| 98 |
+
|
| 99 |
+
## Benchmarks (CEAS-2008, n=2,000 held-out)
|
| 100 |
+
|
| 101 |
+
| Metric | Phish_Byte | DistilBERT fine-tuned |
|
| 102 |
+
|--------|:----------:|:---------------------:|
|
| 103 |
+
| F1 score | **0.948** | ~0.967 |
|
| 104 |
+
| Parameters | **12,545** | 66,000,000 |
|
| 105 |
+
| Model size | **52 KB** | ~250 MB |
|
| 106 |
+
| Throughput (GPU) | **1,527/sec** | ~50/sec |
|
| 107 |
+
| GPU required | **No** | Practically yes |
|
| 108 |
+
| Header analysis | **Yes (SPF, DKIM)** | No |
|
| 109 |
+
| Explainability | **29-feature attribution** | Token-level SHAP |
|
| 110 |
+
|
| 111 |
+
## Feature signals (29 inputs)
|
| 112 |
+
|
| 113 |
+
| Category | Features |
|
| 114 |
+
|----------|----------|
|
| 115 |
+
| Domain (5) | mismatch, Reply-To diff, Return-Path diff, freemail flag, brand impersonation |
|
| 116 |
+
| URL (5) | HTTPS ratio, anchor mismatch, suspicious TLD, urgency, link density |
|
| 117 |
+
| SPF (3) | fail, no record, no sending IP |
|
| 118 |
+
| Subject (7) | urgency, security theme, brand name, currency, all-caps, fake RE, fake transaction ID |
|
| 119 |
+
| Character-level (5) | caps ratio, digit ratio, special density, word length, HTML ratio |
|
| 120 |
+
| Composite (4) | per-layer normalized scores |
|
| 121 |
+
|
| 122 |
## Architecture
|
| 123 |
|
| 124 |
```
|
| 125 |
+
raw email
|
| 126 |
+
β Layer 1 (rule scorers, ~1ms) β confidence gate
|
| 127 |
+
β Layer 2 (custom MLP, ~3ms) β PhishVerdict
|
| 128 |
+
{label, probability, confidence, layer_used, feature_weights}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
```
|
| 130 |
|
| 131 |
+
## Install
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
|
| 133 |
+
```bash
|
| 134 |
+
pip install huggingface_hub safetensors dnspython
|
| 135 |
+
```
|
| 136 |
|
| 137 |
+
```python
|
| 138 |
+
from phishbyte import PhishByteEngine
|
| 139 |
+
engine = PhishByteEngine.from_pretrained("SamSec007/phishbyte")
|
| 140 |
+
verdict = engine.analyze(raw_email_string)
|
| 141 |
+
```
|
|
|
|
| 142 |
|
| 143 |
## Limitations
|
| 144 |
|
| 145 |
+
- ~5% error rate (F1 0.948). Use as one signal in defence-in-depth.
|
| 146 |
+
- Trained on CEAS-2008 (English, 2008-era phishing). Modern attack patterns may reduce recall.
|
| 147 |
+
- SPF validation skipped during training on historical data β re-enables at inference time.
|
|
|
|
| 148 |
|
| 149 |
## Citation
|
| 150 |
|
| 151 |
```bibtex
|
| 152 |
@software{phishbyte2026,
|
| 153 |
+
author = {Singh, Samratth},
|
| 154 |
+
title = {Phish_Byte: Cascading from-scratch PyTorch phishing detection},
|
| 155 |
+
year = {2026},
|
| 156 |
+
url = {https://github.com/AnonymousSingh-007/Phish_Byte}
|
| 157 |
}
|
| 158 |
```
|
| 159 |
|