agentlans/multilingual-e5-small-domain-classifier

multilingual-e5-small Domain Classifier

A fine-tuned version of the bert architecture (BertForSequenceClassification) optimized for the text-classification task.

• Model type: bert
• Problem Type: single_label_classification
• Number of Labels: 26
• Vocabulary Size: 250037
• License: MIT

Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/multilingual-e5-small-domain-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")

Intended Uses & Limitations

Intended Use

This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

Label ID	Label Name
0	Adult
1	Arts_and_Entertainment
2	Autos_and_Vehicles
3	Beauty_and_Fitness
4	Books_and_Literature
5	Business_and_Industrial
6	Computers_and_Electronics
7	Finance
8	Food_and_Drink
9	Games
10	Health
11	Hobbies_and_Leisure
12	Home_and_Garden
13	Internet_and_Telecom
14	Jobs_and_Education
15	Law_and_Government
16	News
17	Online_Communities
18	People_and_Society
19	Pets_and_Animals
20	Real_Estate
21	Science
22	Sensitive_Subjects
23	Shopping
24	Sports
25	Travel_and_Transportation

Training Details

Hyperparameters

The following hyperparameters were used during fine-tuning:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Optimizer: OptimizerNames.ADAMW_TORCH_FUSED
• Number of Epochs: 3.0
• Mixed Precision: BF16

Show Advanced Training Configuration

Optimization & Regularization

• Gradient Accumulation Steps: 1
• Learning Rate Scheduler: SchedulerType.LINEAR
• Warmup Steps: 0
• Warmup Ratio: None
• Weight Decay: 0.0
• Max Gradient Norm: 1.0

Hardware & Reproducibility

• Number of GPUs: 1
• Seed: 42

Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

Metric	Value
Train Loss	0.6686
Validation Loss	0.9974
Validation F1 Score	0.7709
Total FLOPs	7.9086e+15

Speed Performance

• Training Runtime: 1639.5865 seconds
• Train Samples per Second: 292.775
• Evaluation Runtime: 10.8576 seconds
• Eval Samples per Second: 1842.216

Show Detailed Training Logs

Training Logs History

Step	Epoch	Learning Rate	Training Loss	Validation Loss	Validation F1
500	0.025	4.9584e-05	2.602	N/A	N/A
1000	0.05	4.9168e-05	1.8965	N/A	N/A
1500	0.075	4.8751e-05	1.604	N/A	N/A
2000	0.1	4.8334e-05	1.3957	N/A	N/A
2500	0.125	4.7918e-05	1.322	N/A	N/A
3000	0.15	4.7501e-05	1.2218	N/A	N/A
3500	0.175	4.7084e-05	1.195	N/A	N/A
4000	0.2	4.6668e-05	1.1313	N/A	N/A
4500	0.225	4.6251e-05	1.0902	N/A	N/A
5000	0.25	4.5835e-05	1.0637	N/A	N/A
5500	0.275	4.5418e-05	1.0626	N/A	N/A
6000	0.3	4.5001e-05	1.0054	N/A	N/A
6500	0.325	4.4585e-05	1.0253	N/A	N/A
7000	0.35	4.4168e-05	1.0127	N/A	N/A
7500	0.375	4.3751e-05	0.9714	N/A	N/A
8000	0.4	4.3335e-05	0.9589	N/A	N/A
8500	0.425	4.2918e-05	0.9808	N/A	N/A
9000	0.45	4.2502e-05	0.9392	N/A	N/A
9500	0.475	4.2085e-05	0.9304	N/A	N/A
10000	0.5	4.1668e-05	0.9369	N/A	N/A
10500	0.525	4.1252e-05	0.9181	N/A	N/A
11000	0.55	4.0835e-05	0.8996	N/A	N/A
11500	0.575	4.0418e-05	0.9111	N/A	N/A
12000	0.6	4.0002e-05	0.9033	N/A	N/A
12500	0.625	3.9585e-05	0.917	N/A	N/A
13000	0.65	3.9169e-05	0.8872	N/A	N/A
13500	0.675	3.8752e-05	0.8604	N/A	N/A
14000	0.7	3.8335e-05	0.8628	N/A	N/A
14500	0.725	3.7919e-05	0.8929	N/A	N/A
15000	0.75	3.7502e-05	0.8585	N/A	N/A
15500	0.775	3.7085e-05	0.9014	N/A	N/A
16000	0.8	3.6669e-05	0.8581	N/A	N/A
16500	0.825	3.6252e-05	0.8622	N/A	N/A
17000	0.85	3.5836e-05	0.873	N/A	N/A
17500	0.875	3.5419e-05	0.8446	N/A	N/A
18000	0.9	3.5002e-05	0.819	N/A	N/A
18500	0.925	3.4586e-05	0.8458	N/A	N/A
19000	0.95	3.4169e-05	0.8458	N/A	N/A
19500	0.975	3.3752e-05	0.8497	N/A	N/A
20000	1.0	3.3336e-05	0.7989	N/A	N/A
20002	1.0	N/A	N/A	0.8514	0.7452
20500	1.025	3.2919e-05	0.6034	N/A	N/A
21000	1.05	3.2503e-05	0.6148	N/A	N/A
21500	1.075	3.2086e-05	0.614	N/A	N/A
22000	1.1	3.1669e-05	0.5895	N/A	N/A
22500	1.125	3.1253e-05	0.6483	N/A	N/A
23000	1.15	3.0836e-05	0.6331	N/A	N/A
23500	1.175	3.0419e-05	0.5885	N/A	N/A
24000	1.2	3.0003e-05	0.6082	N/A	N/A
24500	1.225	2.9586e-05	0.6312	N/A	N/A
25000	1.25	2.9170e-05	0.6033	N/A	N/A
25500	1.275	2.8753e-05	0.6006	N/A	N/A
26000	1.3	2.8336e-05	0.6283	N/A	N/A
26500	1.325	2.7920e-05	0.6319	N/A	N/A
27000	1.35	2.7503e-05	0.5913	N/A	N/A
27500	1.375	2.7086e-05	0.6037	N/A	N/A
28000	1.4	2.6670e-05	0.6025	N/A	N/A
28500	1.425	2.6253e-05	0.6067	N/A	N/A
29000	1.45	2.5837e-05	0.6075	N/A	N/A
29500	1.475	2.5420e-05	0.6035	N/A	N/A
30000	1.5	2.5003e-05	0.5826	N/A	N/A
30500	1.525	2.4587e-05	0.5905	N/A	N/A
31000	1.55	2.4170e-05	0.563	N/A	N/A
31500	1.575	2.3753e-05	0.5795	N/A	N/A
32000	1.6	2.3337e-05	0.603	N/A	N/A
32500	1.625	2.2920e-05	0.5805	N/A	N/A
33000	1.65	2.2504e-05	0.6108	N/A	N/A
33500	1.675	2.2087e-05	0.6077	N/A	N/A
34000	1.7	2.1670e-05	0.5751	N/A	N/A
34500	1.725	2.1254e-05	0.5833	N/A	N/A
35000	1.75	2.0837e-05	0.5895	N/A	N/A
35500	1.775	2.0420e-05	0.5541	N/A	N/A
36000	1.8	2.0004e-05	0.5423	N/A	N/A
36500	1.825	1.9587e-05	0.5566	N/A	N/A
37000	1.85	1.9171e-05	0.5493	N/A	N/A
37500	1.875	1.8754e-05	0.5602	N/A	N/A
38000	1.9	1.8337e-05	0.5878	N/A	N/A
38500	1.925	1.7921e-05	0.5681	N/A	N/A
39000	1.95	1.7504e-05	0.5464	N/A	N/A
39500	1.975	1.7087e-05	0.5917	N/A	N/A
40000	2.0	1.6671e-05	0.5443	N/A	N/A
40004	2.0	N/A	N/A	0.8536	0.7652
40500	2.025	1.6254e-05	0.3501	N/A	N/A
41000	2.05	1.5838e-05	0.3785	N/A	N/A
41500	2.075	1.5421e-05	0.4034	N/A	N/A
42000	2.1	1.5004e-05	0.385	N/A	N/A
42500	2.125	1.4588e-05	0.3758	N/A	N/A
43000	2.15	1.4171e-05	0.3713	N/A	N/A
43500	2.175	1.3754e-05	0.413	N/A	N/A
44000	2.2	1.3338e-05	0.3787	N/A	N/A
44500	2.225	1.2921e-05	0.3805	N/A	N/A
45000	2.25	1.2505e-05	0.3757	N/A	N/A
45500	2.275	1.2088e-05	0.3887	N/A	N/A
46000	2.3	1.1671e-05	0.3789	N/A	N/A
46500	2.325	1.1255e-05	0.3742	N/A	N/A
47000	2.35	1.0838e-05	0.3805	N/A	N/A
47500	2.375	1.0421e-05	0.3936	N/A	N/A
48000	2.4	1.0005e-05	0.38	N/A	N/A
48500	2.425	9.5882e-06	0.3941	N/A	N/A
49000	2.45	9.1716e-06	0.4054	N/A	N/A
49500	2.475	8.7550e-06	0.3659	N/A	N/A
50000	2.5	8.3383e-06	0.3917	N/A	N/A
50500	2.525	7.9217e-06	0.3876	N/A	N/A
51000	2.55	7.5051e-06	0.3628	N/A	N/A
51500	2.575	7.0885e-06	0.3918	N/A	N/A
52000	2.6	6.6718e-06	0.359	N/A	N/A
52500	2.625	6.2552e-06	0.3634	N/A	N/A
53000	2.65	5.8386e-06	0.3737	N/A	N/A
53500	2.675	5.4220e-06	0.4022	N/A	N/A
54000	2.7	5.0053e-06	0.3562	N/A	N/A
54500	2.725	4.5887e-06	0.349	N/A	N/A
55000	2.75	4.1721e-06	0.3573	N/A	N/A
55500	2.775	3.7555e-06	0.335	N/A	N/A
56000	2.8	3.3388e-06	0.3679	N/A	N/A
56500	2.825	2.9222e-06	0.3266	N/A	N/A
57000	2.85	2.5056e-06	0.3453	N/A	N/A
57500	2.875	2.0890e-06	0.3682	N/A	N/A
58000	2.9	1.6723e-06	0.3417	N/A	N/A
58500	2.925	1.2557e-06	0.3192	N/A	N/A
59000	2.95	8.3908e-07	0.3375	N/A	N/A
59500	2.975	4.2246e-07	0.3669	N/A	N/A
60000	3.0	5.8328e-09	0.332	N/A	N/A
60006	3.0	N/A	N/A	0.9974	0.7709

Framework Versions

• Transformers: 5.0.0.dev0
• PyTorch: 2.9.1+cu128