GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech

mmBERT: Advanced Multilingual Encoder Model

Model Description

This model is a multilingual hate speech detection model fine-tuned from jhu-clsp/mmBERT-base (mmBERT) using datasets from 21 languages belonging to multiple language families and writing systems. The model aims to support robust multilingual hate speech classification and cross-lingual generalization across both high-resource and low-resource languages.

The model performs binary classification:

• Hate Speech
• Non-Hate Speech

Model Details

• Developed by: Ghadeer Albadani
• Base Model: bert-base-multilingual-cased (mBERT)
• Model Type: Transformer-based Text Classification
• Task: Multilingual Hate Speech Detection
• Framework: Hugging Face Transformers
• Training Languages: 21 Languages
• License: Apache-2.0

Supported Languages

The model was fine-tuned using the following languages:

• Arabic
• Hebrew
• Persian
• English
• French
• German
• Spanish
• Portuguese
• Italian
• Danish
• Russian
• Turkish
• Bengali
• Chinese
• Korean
• Malay
• Indonesian
• Swahili
• Amharic
• Somali
• Roman Urdu

These languages represent diverse language families and writing systems, enabling multilingual hate speech detection and cross-lingual generalization.

Intended Use

Direct Use

The model can be used for:

• Multilingual hate speech detection
• Toxic content classification
• Social media moderation
• Multilingual NLP research
• Cross-lingual text classification

Downstream Applications

• Content moderation systems
• Hate speech monitoring platforms
• Social media analytics
• Cross-lingual NLP applications
• Low-resource language research

Out-of-Scope Use

This model should not be used as:

• A legal decision-making system
• A replacement for human moderation
• A profiling tool for individuals or groups
• A fully automated moderation system without human oversight

Benchmark Performance

Evaluation Results

The model was evaluated independently on multilingual hate speech datasets covering 20 languages.

Language	Accuracy	F1-Score	Notes
Arabic	0.82	0.82	Moderate performance
Persian	0.94	0.94	Excellent performance
Hebrew	0.76	0.76	Lower performance
Bengali	0.89	0.89	Good performance
Korean	0.76	0.76	Lower performance
Chinese	0.81	0.81	Moderate performance
Russian	0.89	0.89	Good performance
Spanish	0.82	0.82	Moderate performance
Indonesian	0.94	0.94	Excellent performance
Turkish	0.85	0.85	Good performance
English	0.89	0.89	Good performance
French	0.85	0.85	Good performance
German	0.64	0.63	Poor performance
Portuguese	0.71	0.71	Moderate–poor performance
Malay	0.65	0.65	Poor performance
Italian	0.80	0.79	Moderate performance
Roman Urdu	0.82	0.82	Moderate performance
Amharic	0.77	0.76	Moderate performance
Swahili	0.90	0.90	Very good performance
Somali	0.71	0.71	Moderate–poor performance

Summary

The model demonstrated strong multilingual hate speech detection capabilities across a diverse set of languages. The highest performance was achieved on Persian and Indonesian, both obtaining an Accuracy and F1-score of 0.94, followed by Swahili (0.90), English (0.89), Russian (0.89), and Bengali (0.89). These results indicate that the model successfully learned language-independent hate speech representations despite substantial linguistic diversity among the training languages.

Training Data

The model was fine-tuned on multilingual hate speech datasets collected from multiple publicly available sources covering 21 languages.

The datasets contain two labels:

• Hate Speech
• Non-Hate Speech

Data preprocessing included cleaning, normalization, tokenization, and label standardization.

Training Procedure

Hyperparameters

Parameter	Value
Epochs	2
Learning Rate	2e-5
Batch Size	32
Optimizer Epsilon	1e-8
Maximum Sequence Length	512

Hardware

Training was performed using:

• GPU: NVIDIA L4
• GPU Memory: 24 GB

Software

• Python
• PyTorch
• Hugging Face Transformers
• CUDA

Evaluation Metrics

The model was evaluated using:

• Accuracy
• Precision
• Recall
• F1-Score
• Macro F1

Bias, Risks, and Limitations

Although the model was trained on multilingual datasets from diverse language families, performance may vary depending on:

• Dataset quality
• Annotation consistency
• Cultural interpretation of hate speech
• Domain differences
• Language-specific characteristics

Users should evaluate the model carefully before deployment in real-world moderation systems.

How to Use

from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
from transformers import pipeline

model_name = "GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

classifier = pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer
)

text = "I hate all people from this group."

result = classifier(text)

print(result)

Model Architecture

The model is based on (mBERT).

Architecture details:

-Transformer Encoder Architecture -22 Transformer Layers -Hidden Size: 768 -Intermediate Size: 1152 -12 Attention Heads -Approximately 307 Million Parameters -110 Million Non-embedding Parameters -Maximum Sequence Length: 8192 Tokens -Vocabulary Size: 256,000 Tokens -Gemma 2 Tokenizer -Pretrained on multilingual web data, Wikipedia, academic papers, code repositories, and community discussions -Supports multilingual understanding and cross-lingual transfer learning

Research Context

This model was developed as part of research on multilingual hate speech detection, cross-lingual transfer learning, and multilingual natural language processing.

The research investigates:

• Multilingual training
• Cross-lingual transfer
• Zero-shot learning
• Hate speech detection in low-resource languages
• Language-independent representation learning

Citation

If you use this model in your research, please cite:

@misc{albadani2026mmbert,
  author = {Ghadeer Albadani},
  title = {mmBERT: Multilingual Detection of Hate Speech},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech}
}

Contact

Author: Ghadeer Albadani

Model Repository: https://huggingface.co/GhadeerALbadani/mmbert-Multilingual_detection_of_hate_speech