agentlans/multilingual-e5-small-doc-type-v2-classifier

multilingual-e5-small Document Type V2 Classifier

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 25
• Vocabulary Size: 250037
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/multilingual-e5-small-doc-type-v2-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Label ID	Label Name
0	About (Org.)
1	About (Personal)
2	Academic Writing
3	Audio Transcript
4	Comment Section
5	Content Listing
6	Creative Writing
7	Customer Support
8	Documentation
9	FAQ
10	Knowledge Article
11	Legal Notices
12	Listicle
13	News (Org.)
14	News Article
15	Nonfiction Writing
16	Other/Unclassified
17	Personal Blog
18	Product Page
19	Q&A Forum
20	Spam / Ads
21	Structured Data
22	Truncated
23	Tutorial
24	User Review

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

Show Advanced Training Configuration

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

Metric	Value
Train Loss	0.5709
Validation Loss	0.8624
Validation F1 Score	0.809
Total FLOPs	7.9082e+15

Speed Performance

• Training Runtime: 1693.148 seconds
• Train Samples per Second: 283.503
• Evaluation Runtime: 11.4879 seconds
• Eval Samples per Second: 1741.655

Show Detailed Training Logs

Training Logs History

Step	Epoch	Learning Rate	Training Loss	Validation Loss	Validation F1
500	0.025	4.9584e-05	1.8537	N/A	N/A
1000	0.05	4.9168e-05	1.3289	N/A	N/A
1500	0.075	4.8751e-05	1.1698	N/A	N/A
2000	0.1	4.8334e-05	1.0996	N/A	N/A
2500	0.125	4.7918e-05	1.0552	N/A	N/A
3000	0.15	4.7501e-05	1.0462	N/A	N/A
3500	0.175	4.7084e-05	1.0004	N/A	N/A
4000	0.2	4.6668e-05	0.9812	N/A	N/A
4500	0.225	4.6251e-05	0.9245	N/A	N/A
5000	0.25	4.5834e-05	0.9282	N/A	N/A
5500	0.275	4.5418e-05	0.9167	N/A	N/A
6000	0.3	4.5001e-05	0.8886	N/A	N/A
6500	0.325	4.4584e-05	0.8826	N/A	N/A
7000	0.35	4.4168e-05	0.8443	N/A	N/A
7500	0.375	4.3751e-05	0.8374	N/A	N/A
8000	0.4	4.3334e-05	0.8271	N/A	N/A
8500	0.425	4.2918e-05	0.8306	N/A	N/A
9000	0.45	4.2501e-05	0.8561	N/A	N/A
9500	0.475	4.2085e-05	0.7851	N/A	N/A
10000	0.5	4.1668e-05	0.7841	N/A	N/A
10500	0.525	4.1251e-05	0.7678	N/A	N/A
11000	0.55	4.0835e-05	0.7538	N/A	N/A
11500	0.575	4.0418e-05	0.735	N/A	N/A
12000	0.6	4.0001e-05	0.774	N/A	N/A
12500	0.625	3.9585e-05	0.7368	N/A	N/A
13000	0.65	3.9168e-05	0.7435	N/A	N/A
13500	0.675	3.8751e-05	0.7035	N/A	N/A
14000	0.7	3.8335e-05	0.7552	N/A	N/A
14500	0.725	3.7918e-05	0.7443	N/A	N/A
15000	0.75	3.7501e-05	0.7461	N/A	N/A
15500	0.775	3.7085e-05	0.7352	N/A	N/A
16000	0.8	3.6668e-05	0.6946	N/A	N/A
16500	0.825	3.6252e-05	0.6939	N/A	N/A
17000	0.85	3.5835e-05	0.7509	N/A	N/A
17500	0.875	3.5418e-05	0.6992	N/A	N/A
18000	0.9	3.5002e-05	0.7043	N/A	N/A
18500	0.925	3.4585e-05	0.6977	N/A	N/A
19000	0.95	3.4168e-05	0.6952	N/A	N/A
19500	0.975	3.3752e-05	0.708	N/A	N/A
20000	1.0	3.3335e-05	0.6695	N/A	N/A
20001	1.0	N/A	N/A	0.6958	0.7876
20500	1.025	3.2918e-05	0.5363	N/A	N/A
21000	1.05	3.2502e-05	0.547	N/A	N/A
21500	1.075	3.2085e-05	0.5733	N/A	N/A
22000	1.1	3.1668e-05	0.5454	N/A	N/A
22500	1.125	3.1252e-05	0.5235	N/A	N/A
23000	1.15	3.0835e-05	0.5291	N/A	N/A
23500	1.175	3.0418e-05	0.5537	N/A	N/A
24000	1.2	3.0002e-05	0.555	N/A	N/A
24500	1.225	2.9585e-05	0.5338	N/A	N/A
25000	1.25	2.9169e-05	0.5615	N/A	N/A
25500	1.275	2.8752e-05	0.5155	N/A	N/A
26000	1.3	2.8335e-05	0.5353	N/A	N/A
26500	1.325	2.7919e-05	0.5317	N/A	N/A
27000	1.35	2.7502e-05	0.5429	N/A	N/A
27500	1.375	2.7085e-05	0.5311	N/A	N/A
28000	1.4	2.6669e-05	0.5345	N/A	N/A
28500	1.425	2.6252e-05	0.5287	N/A	N/A
29000	1.45	2.5835e-05	0.5204	N/A	N/A
29500	1.475	2.5419e-05	0.5121	N/A	N/A
30000	1.5	2.5002e-05	0.52	N/A	N/A
30500	1.525	2.4585e-05	0.5094	N/A	N/A
31000	1.55	2.4169e-05	0.5169	N/A	N/A
31500	1.575	2.3752e-05	0.5226	N/A	N/A
32000	1.6	2.3335e-05	0.5281	N/A	N/A
32500	1.625	2.2919e-05	0.5246	N/A	N/A
33000	1.65	2.2502e-05	0.532	N/A	N/A
33500	1.675	2.2086e-05	0.5068	N/A	N/A
34000	1.7	2.1669e-05	0.4971	N/A	N/A
34500	1.725	2.1252e-05	0.5122	N/A	N/A
35000	1.75	2.0836e-05	0.489	N/A	N/A
35500	1.775	2.0419e-05	0.479	N/A	N/A
36000	1.8	2.0002e-05	0.4919	N/A	N/A
36500	1.825	1.9586e-05	0.4974	N/A	N/A
37000	1.85	1.9169e-05	0.5045	N/A	N/A
37500	1.875	1.8752e-05	0.525	N/A	N/A
38000	1.9	1.8336e-05	0.4748	N/A	N/A
38500	1.925	1.7919e-05	0.4831	N/A	N/A
39000	1.95	1.7502e-05	0.5091	N/A	N/A
39500	1.975	1.7086e-05	0.4821	N/A	N/A
40000	2.0	1.6669e-05	0.4862	N/A	N/A
40002	2.0	N/A	N/A	0.7491	0.797
40500	2.025	1.6253e-05	0.357	N/A	N/A
41000	2.05	1.5836e-05	0.333	N/A	N/A
41500	2.075	1.5419e-05	0.374	N/A	N/A
42000	2.1	1.5003e-05	0.3698	N/A	N/A
42500	2.125	1.4586e-05	0.3759	N/A	N/A
43000	2.15	1.4169e-05	0.3543	N/A	N/A
43500	2.175	1.3753e-05	0.3695	N/A	N/A
44000	2.2	1.3336e-05	0.3385	N/A	N/A
44500	2.225	1.2919e-05	0.3583	N/A	N/A
45000	2.25	1.2503e-05	0.3445	N/A	N/A
45500	2.275	1.2086e-05	0.3575	N/A	N/A
46000	2.3	1.1669e-05	0.3382	N/A	N/A
46500	2.325	1.1253e-05	0.3732	N/A	N/A
47000	2.35	1.0836e-05	0.3454	N/A	N/A
47500	2.375	1.0419e-05	0.3563	N/A	N/A
48000	2.4	1.0003e-05	0.3302	N/A	N/A
48500	2.425	9.5862e-06	0.3421	N/A	N/A
49000	2.45	9.1695e-06	0.3119	N/A	N/A
49500	2.475	8.7529e-06	0.3578	N/A	N/A
50000	2.5	8.3362e-06	0.3584	N/A	N/A
50500	2.525	7.9196e-06	0.3142	N/A	N/A
51000	2.55	7.5030e-06	0.3124	N/A	N/A
51500	2.575	7.0863e-06	0.3262	N/A	N/A
52000	2.6	6.6697e-06	0.3072	N/A	N/A
52500	2.625	6.2530e-06	0.3274	N/A	N/A
53000	2.65	5.8364e-06	0.3131	N/A	N/A
53500	2.675	5.4197e-06	0.3281	N/A	N/A
54000	2.7	5.0031e-06	0.3108	N/A	N/A
54500	2.725	4.5864e-06	0.3189	N/A	N/A
55000	2.75	4.1698e-06	0.3367	N/A	N/A
55500	2.775	3.7531e-06	0.2969	N/A	N/A
56000	2.8	3.3365e-06	0.3332	N/A	N/A
56500	2.825	2.9199e-06	0.3197	N/A	N/A
57000	2.85	2.5032e-06	0.312	N/A	N/A
57500	2.875	2.0866e-06	0.3275	N/A	N/A
58000	2.9	1.6699e-06	0.2933	N/A	N/A
58500	2.925	1.2533e-06	0.3123	N/A	N/A
59000	2.95	8.3662e-07	0.3045	N/A	N/A
59500	2.975	4.1998e-07	0.2928	N/A	N/A
60000	3.0	3.3332e-09	0.3199	N/A	N/A
60003	3.0	N/A	N/A	0.8624	0.809

Framework Versions

• Transformers: 5.0.0.dev0
• PyTorch: 2.9.1+cu128