AtmicReRanker (MonoBERT cross-encoder)

LambdaRank-weighted pairwise fine-tune of castorini/monobert-large-msmarco on the combined layer1+layer2 Ramana Maharshi Q&A reranking dataset (7,328 training queries / 1,280 validation queries), max_length=512.

BREAKING CHANGE: this repo previously held a sentence-transformers-compatible bge-reranker-base CrossEncoder checkpoint. It now holds a legacy PyGaggle-style MonoBERT checkpoint (BertConfig+BertForSequenceClassification, num_labels=2), which is not loadable via sentence_transformers.cross_encoder.CrossEncoder.

How to load

Loads exactly the same way as the base model it was fine-tuned from — the only difference is the repo id (and therefore the weights); architecture, tokenizer format, and scoring convention are identical, so you can swap between base and fine-tuned by changing one string:

from transformers import BertConfig, BertForSequenceClassification, BertTokenizerFast

# swap this for "castorini/monobert-large-msmarco" to compare against base
repo_id = "SriRamanaAtmic/AtmicReRanker"

cfg = BertConfig.from_pretrained(repo_id, num_labels=2)
model = BertForSequenceClassification.from_pretrained(repo_id, config=cfg)
tok = BertTokenizerFast.from_pretrained(repo_id)  # NOT BertTokenizer — this repo has no vocab.txt,
                                                   # only tokenizer.json; BertTokenizerFast handles both

enc = tok([[query, passage]], padding=True, truncation="only_second", max_length=512, return_tensors="pt")
logits = model(**enc).logits
relevance_score = logits[:, 1] - logits[:, 0]   # higher = more relevant
# or: relevance_prob = logits.softmax(-1)[:, 1]

truncation="only_second" (not truncation=True) is deliberate: it guarantees only the passage is ever truncated, never the query, and avoids a tokenizer warning that "longest_first" (what truncation=True maps to) raises for sequence pairs.

Training

base model	castorini/monobert-large-msmarco
loss	LambdaRank-weighted pairwise: RankNet logistic loss on the relevance-score margin (logits[:,1]-logits[:,0]), scaled per pair by |ΔNDCG| from that pair's rank-swap impact
frozen layers	embeddings + bottom 20 of 24 BERT-large encoder layers (top 4 + pooler + classifier trainable)
learning rate	2e-5
batch size / grad accum	32 / 1 (effective batch 32)
max sequence length	512
epochs tried	2
checkpoint selection	smallest |train_loss − val_loss| gap, checked every 50 steps (not just per-epoch) — best found at step 40, well inside epoch 1, before the gap widened further into training

The train/val-loss-gap selection criterion (rather than best-epoch-only or a pure accuracy proxy) was chosen specifically to catch overfitting early: an earlier training run on this same data that selected checkpoints by validation pairwise-accuracy alone kept improving on that narrow metric through 4 full epochs, but regressed below the base model on the real closed-pool benchmark below — it had overfit to the training distribution's negative style in a way the accuracy proxy didn't detect. Selecting by loss gap and checking sub-epoch caught that failure mode and avoided it.

Benchmark

Closed-pool ranking on the combined layer1+layer2 validation set (1,280 queries, 1,072-passage corpus), using SriRamanaAtmic/AtmicEmbeddingv3 as the stage-1 retriever (top-20 candidates per query reranked; candidates outside top-20 count as a miss for every metric — stage1_ceil below is the ceiling this caps both rerankers at).

Metric	stage1 dense alone	base MonoBERT	AtmicReRanker (this model)
recall@1	0.3406	0.4766	0.4961
recall@5	0.5508	0.6336	0.6461
mrr@10	0.4324	0.5450	0.5604
ndcg@10	0.4794	0.5787	0.5902

stage1_ceil (top-20 recall, caps both rerankers): 0.7156

This fine-tune improves on base MonoBERT across every metric on this domain's validation set.