nuha-multiclass

Model Summary

nuha-multiclass is an Arabic text classifier that categorises Jordanian social media comments into three classes based on the NUHA methodology for online gender-based violence (OGBV). It fine-tunes nuha-mlm — a domain-adapted Arabic BERT — and outputs one of:

Label	Meaning
Not Online Violence	Comments that are not hate speech
Offensive Language	Hate speech characterised by irony or sarcasm
Gender Based Violence	Direct hate speech targeting gender — the primary focus of NUHA

This model was developed as part of a pilot proof-of-concept for the NUHA project by the Jordan Open Source Association (JOSA). It is the production model behind the NUHA analysis platform.

A lightweight, ONNX-optimised 4-layer classifier trained on the same task is available at thejosango/nuha.

For a simpler binary classifier (hate speech / non-hate speech), see nuha-binary.

Uses

Direct Use

Classifying Arabic social media comments for online gender-based violence, particularly for Jordanian Arabic content from Facebook and X (Twitter).

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="thejosango/nuha-multiclass",
    tokenizer="thejosango/nuha-multiclass",
)

result = classifier("اخرسي يا غبية")
print(result)
# [{'label': 'Gender Based Violence', 'score': ...}]

For batch inference:

comments = ["يعطيكم العافية", "أنتِ ساحرة", "اخرسي يا غبية"]
results = classifier(comments)
for comment, result in zip(comments, results):
    print(f"{result['label']} ({result['score']:.2f}): {comment}")

Using the ONNX Version

For faster CPU inference, use the ONNX export:

from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline

model = ORTModelForSequenceClassification.from_pretrained("thejosango/nuha")
tokenizer = AutoTokenizer.from_pretrained("thejosango/nuha")
classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)

Out-of-Scope Use

• Other Arabic dialects: The model was trained primarily on Jordanian Arabic. Performance on Egyptian, Gulf, or Modern Standard Arabic is not validated.
• Other hate speech targets: NUHA is calibrated for online gender-based violence. It is not designed to detect hate speech targeting race, religion, or other demographics.
• High-stakes automated decisions: Given the moderate performance (F1 ≈ 0.54) and pilot nature of this work, the model should not be used as the sole decision-maker in content moderation systems without human review.

Bias, Risks, and Limitations

• Pilot annotation quality: Training labels were produced in an exploratory annotation effort with variable inter-annotator agreement. The model inherits noise from that process, which is reflected in the moderate F1 score.
• Three-class difficulty: Distinguishing Offensive Language from Gender Based Violence is a genuinely difficult subtask. The Offensive Language class is small (≈2% of training data) and the model may struggle with it.
• Colloquial Arabic only: The aggressive text cleaning (Arabic-only filtering) means the model has never seen URLs, numbers, punctuation, or Latin-script text.
• Imbalanced classes: The training data is dominated by Not Online Violence (≈59%), with Offensive Language being very sparse (≈2%). Data augmentation was applied but class imbalance remains a factor.

Training Details

Training Data

Fine-tuned on the methodology configuration of thejosango/nuha-dataset, which applies the three-class NUHA categorisation scheme to the original annotations.

Preprocessing

At training and inference time, the following normalisation is applied to input text (in addition to the dataset-level Arabic-only filtering):

1. URLs replaced with [رابط] token
2. @mentions replaced with [مستخدم] token
3. Email addresses replaced with [بريد] token
4. Numbers removed
5. Punctuation removed
6. Arabic diacritics (harakat) removed
7. Whitespace normalised

Hyperparameters

Parameter	Value
Base model	thejosango/nuha-mlm
Hidden layers	12 (full depth)
Learning rate	5e-5
LR schedule	Constant
Batch size	64
Epochs	5
Weight decay	0.0
Label smoothing	0.1
Weighted loss	No
Data augmentation	Yes (contextual word substitution, ratio 0.75)
Framework	Transformers 4.32.1, PyTorch 2.0.1

Evaluation Results

Evaluated on the validation split of thejosango/nuha-dataset (methodology configuration):

Metric	Value
F1 (macro)	0.5363
Precision	0.6660
Recall	0.5188
Loss	0.7126

The lower recall relative to precision suggests the model is conservative — it tends to under-predict Gender Based Violence rather than over-predict it. This reflects both the difficulty of the three-class task and the limited size of the pilot training corpus.

---

This model was developed as part of an initial pilot study. Performance metrics reflect the complexity of the task and the proof-of-concept nature of this system.