Zero-Shot Text Classification — projection_biencoder

CLIP-inspired with projection heads, L2 norm, and learnable temperature.

This model encodes texts and candidate labels into a shared embedding space using BERT, enabling classification into arbitrary categories without retraining for new labels.

Training Details

Parameter	Value
Base model	bert-base-uncased
Model variant	projection_biencoder
Training steps	1000
Batch size	2
Learning rate	2e-05
Trainable params	109,679,105
Training time	318.3s

Dataset

Trained on polodealvarado/zeroshot-classification.

Evaluation Results

Metric	Score
Precision	0.9431
Recall	0.9826
F1 Score	0.9624

Usage

from models.projection import ProjectionBiEncoderModel

model = ProjectionBiEncoderModel.from_pretrained("polodealvarado/projection_biencoder")

predictions = model.predict(
    texts=["The stock market crashed yesterday."],
    labels=[["Finance", "Sports", "Biology", "Economy"]],
)
print(predictions)
# [{"text": "...", "scores": {"Finance": 0.98, "Economy": 0.85, ...}}]