SamSec007 commited on
Commit
bea2f62
Β·
verified Β·
1 Parent(s): deec82a

Expand tags, add widget examples, add metrics block

Browse files
Files changed (1) hide show
  1. README.md +86 -54
README.md CHANGED
@@ -1,19 +1,31 @@
1
  ---
2
- language: en
 
3
  license: mit
4
  library_name: phishbyte
5
  pipeline_tag: text-classification
6
  tags:
7
  - phishing-detection
8
  - email-security
 
 
9
  - pytorch
10
  - from-scratch
11
  - no-pretrained-weights
12
  - cascading-inference
13
  - lightweight
14
  - explainable-ai
 
 
 
 
 
 
 
 
 
15
  datasets:
16
- - CEAS-2008
17
  metrics:
18
  - f1
19
  - precision
@@ -31,12 +43,21 @@ model-index:
31
  metrics:
32
  - type: f1
33
  value: 0.948
 
34
  - type: accuracy
35
  value: 0.944
 
36
  - type: precision
37
- value: 0.954
 
38
  - type: recall
39
- value: 0.943
 
 
 
 
 
 
40
  ---
41
 
42
  # Phish_Byte
@@ -45,83 +66,94 @@ A from-scratch PyTorch model for **email phishing detection**.
45
  **F1 0.948** on CEAS-2008. **12,545 parameters** (β‰ˆ9,000Γ— smaller than DistilBERT).
46
  **1,500+ emails/sec** on a laptop GPU. Every verdict explains itself.
47
 
48
- ## Why this exists
49
-
50
- Every phishing detection model on HuggingFace is currently a fine-tuned
51
- transformer (DistilBERT, BERT, RoBERTa) β€” 65 to 110 million parameters,
52
- ~250 MB on disk, ~50 ms per email on GPU. Phish_Byte takes a different
53
- bet: a small custom MLP trained from scratch, fed by 29 carefully chosen
54
- features, routed through a cascading inference pipeline. The model is
55
- **9,000Γ— smaller** than DistilBERT, performs competitively, deploys
56
- without a GPU, and explains every decision.
57
 
58
- ## Usage
59
 
60
  ```python
61
  from phishbyte import PhishByteEngine
62
 
63
- engine = PhishByteEngine.from_pretrained("AnonymousSingh-007/phishbyte")
64
  verdict = engine.analyze(raw_email_string)
65
 
66
  print(verdict.label) # 'phishing'
67
  print(verdict.probability) # 0.9735
68
  print(verdict.confidence) # 'high'
69
- print(verdict.layer_used) # 2 β€” MLP made this call
70
- print(verdict.feature_weights) # full per-feature attribution
71
  ```
72
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
  ## Architecture
74
 
75
  ```
76
- Layer 1 β€” rule scorers (~1 ms): domain + URL + SPF + subject
77
- β”‚
78
- β”œβ”€β”€β–Ί obvious phishing? short-circuit verdict
79
- β”‚
80
- └──► otherwise route to MLP
81
- β”‚
82
- Layer 2 β€” MLP (~3 ms): 29 β†’ 96 β†’ 48 β†’ 1 (sigmoid)
83
- β”‚
84
- β–Ό
85
- PhishVerdict {label, probability, confidence, layer_used, feature_weights}
86
  ```
87
 
88
- ## Performance (CEAS-2008, n=2000 held-out)
89
-
90
- | Metric | Value |
91
- |------------------|----------:|
92
- | F1 score | **0.948** |
93
- | Accuracy | 94.40% |
94
- | Precision | 0.9537 |
95
- | Recall | 0.9432 |
96
- | Parameters | 12,545 |
97
- | Model size | ~50 KB |
98
- | Throughput (GPU) | 1,527 /s |
99
- | Throughput (CPU) | ~800 /s |
100
 
101
- ## Features (29 inputs)
 
 
102
 
103
- - **Domain (5)**: From/Reply-To/Return-Path mismatch, freemail flag, brand impersonation
104
- - **URL (5)**: HTTPS ratio, anchor mismatch, suspicious TLD, urgency, link density
105
- - **SPF (3)**: SPF fail, no record, no sending IP
106
- - **Subject (7)**: urgency, security theme, brand name, currency, all caps, fake RE, fake transaction ID
107
- - **Character-level (5)**: caps ratio, digit ratio, special chars, avg word length, HTML/text ratio
108
- - **Composite (4)**: per-layer normalized scores
109
 
110
  ## Limitations
111
 
112
- - ~5% of decisions are wrong (F1 0.948, not 1.0). Use as one signal in defence-in-depth, not the only gate.
113
- - Trained on CEAS-2008 β€” English-language phishing from 2008. Modern attack patterns and non-English emails will degrade performance.
114
- - SPF validation is bypassed for training (historical domains don't resolve) but runs live at inference time.
115
- - Adversarial emails crafted specifically to game these features will get through.
116
 
117
  ## Citation
118
 
119
  ```bibtex
120
  @software{phishbyte2026,
121
- author = {Singh, Samratth},
122
- title = {Phish_Byte: A cascading from-scratch PyTorch model for email phishing detection},
123
- year = {2026},
124
- url = {https://github.com/AnonymousSingh-007/Phish_Byte}
125
  }
126
  ```
127
 
 
1
  ---
2
+ language:
3
+ - en
4
  license: mit
5
  library_name: phishbyte
6
  pipeline_tag: text-classification
7
  tags:
8
  - phishing-detection
9
  - email-security
10
+ - cybersecurity
11
+ - security
12
  - pytorch
13
  - from-scratch
14
  - no-pretrained-weights
15
  - cascading-inference
16
  - lightweight
17
  - explainable-ai
18
+ - nlp
19
+ - phishing
20
+ - spam-detection
21
+ - malware-detection
22
+ - threat-detection
23
+ - email-classification
24
+ - text-classification
25
+ - feature-engineering
26
+ - interpretable-ml
27
  datasets:
28
+ - ceas-2008
29
  metrics:
30
  - f1
31
  - precision
 
43
  metrics:
44
  - type: f1
45
  value: 0.948
46
+ name: F1 Score
47
  - type: accuracy
48
  value: 0.944
49
+ name: Accuracy
50
  - type: precision
51
+ value: 0.9537
52
+ name: Precision
53
  - type: recall
54
+ value: 0.9432
55
+ name: Recall
56
+ widget:
57
+ - text: "From: PayPal Security <security@paypa1-alert.tk>\nReply-To: attacker@evil-domain.ru\nSubject: URGENT: Your account will be suspended\n\nDear Customer, your PayPal account has been suspended. Verify now at http://paypal-login.tk/verify"
58
+ example_title: "Phishing email example"
59
+ - text: "From: alice@company.com\nReply-To: alice@company.com\nSubject: Team lunch tomorrow\n\nHi everyone, lunch is at noon tomorrow in the usual spot. See you there!"
60
+ example_title: "Legitimate email example"
61
  ---
62
 
63
  # Phish_Byte
 
66
  **F1 0.948** on CEAS-2008. **12,545 parameters** (β‰ˆ9,000Γ— smaller than DistilBERT).
67
  **1,500+ emails/sec** on a laptop GPU. Every verdict explains itself.
68
 
69
+ > **v3 in progress:** expanding to 50K parameters + 6-dataset corpus training.
 
 
 
 
 
 
 
 
70
 
71
+ ## Quick start
72
 
73
  ```python
74
  from phishbyte import PhishByteEngine
75
 
76
+ engine = PhishByteEngine.from_pretrained("SamSec007/phishbyte")
77
  verdict = engine.analyze(raw_email_string)
78
 
79
  print(verdict.label) # 'phishing'
80
  print(verdict.probability) # 0.9735
81
  print(verdict.confidence) # 'high'
82
+ print(verdict.layer_used) # 2
83
+ print(verdict.feature_weights) # per-feature attribution
84
  ```
85
 
86
+ ## Why this exists
87
+
88
+ Every phishing detection model on HuggingFace is a fine-tuned transformer β€”
89
+ DistilBERT, BERT, RoBERTa. 65–110M parameters. ~250 MB on disk. ~50 ms/email.
90
+
91
+ Phish_Byte is different:
92
+ - Custom MLP trained **from scratch** β€” no pretrained weights
93
+ - **29 engineered features** across domain, URL, SPF, subject, and character-level signals
94
+ - **Cascading inference** β€” cheap rules handle obvious cases, MLP handles the rest
95
+ - **Full email header analysis** including live SPF validation
96
+ - Runs on **CPU without a GPU**
97
+ - Every verdict includes **which signals fired and why**
98
+
99
+ ## Benchmarks (CEAS-2008, n=2,000 held-out)
100
+
101
+ | Metric | Phish_Byte | DistilBERT fine-tuned |
102
+ |--------|:----------:|:---------------------:|
103
+ | F1 score | **0.948** | ~0.967 |
104
+ | Parameters | **12,545** | 66,000,000 |
105
+ | Model size | **52 KB** | ~250 MB |
106
+ | Throughput (GPU) | **1,527/sec** | ~50/sec |
107
+ | GPU required | **No** | Practically yes |
108
+ | Header analysis | **Yes (SPF, DKIM)** | No |
109
+ | Explainability | **29-feature attribution** | Token-level SHAP |
110
+
111
+ ## Feature signals (29 inputs)
112
+
113
+ | Category | Features |
114
+ |----------|----------|
115
+ | Domain (5) | mismatch, Reply-To diff, Return-Path diff, freemail flag, brand impersonation |
116
+ | URL (5) | HTTPS ratio, anchor mismatch, suspicious TLD, urgency, link density |
117
+ | SPF (3) | fail, no record, no sending IP |
118
+ | Subject (7) | urgency, security theme, brand name, currency, all-caps, fake RE, fake transaction ID |
119
+ | Character-level (5) | caps ratio, digit ratio, special density, word length, HTML ratio |
120
+ | Composite (4) | per-layer normalized scores |
121
+
122
  ## Architecture
123
 
124
  ```
125
+ raw email
126
+ β†’ Layer 1 (rule scorers, ~1ms) β†’ confidence gate
127
+ β†’ Layer 2 (custom MLP, ~3ms) β†’ PhishVerdict
128
+ {label, probability, confidence, layer_used, feature_weights}
 
 
 
 
 
 
129
  ```
130
 
131
+ ## Install
 
 
 
 
 
 
 
 
 
 
 
132
 
133
+ ```bash
134
+ pip install huggingface_hub safetensors dnspython
135
+ ```
136
 
137
+ ```python
138
+ from phishbyte import PhishByteEngine
139
+ engine = PhishByteEngine.from_pretrained("SamSec007/phishbyte")
140
+ verdict = engine.analyze(raw_email_string)
141
+ ```
 
142
 
143
  ## Limitations
144
 
145
+ - ~5% error rate (F1 0.948). Use as one signal in defence-in-depth.
146
+ - Trained on CEAS-2008 (English, 2008-era phishing). Modern attack patterns may reduce recall.
147
+ - SPF validation skipped during training on historical data β€” re-enables at inference time.
 
148
 
149
  ## Citation
150
 
151
  ```bibtex
152
  @software{phishbyte2026,
153
+ author = {Singh, Samratth},
154
+ title = {Phish_Byte: Cascading from-scratch PyTorch phishing detection},
155
+ year = {2026},
156
+ url = {https://github.com/AnonymousSingh-007/Phish_Byte}
157
  }
158
  ```
159