ConvMemory v2 Evidence Reranker
This is the ConvMemory v0.5.0 protected top-10 token-evidence reranker checkpoint.
It is intended to be attached to the base ConvMemory LoCoMo/MPNet checkpoint:
Purdy0228/ConvMemory-LoCoMo-MPNet.
Usage
from convmemory import ConvMemory
model = ConvMemory.from_pretrained("Purdy0228/ConvMemory-LoCoMo-MPNet")
model.load_evidence_reranker("Purdy0228/ConvMemory-v2-Evidence-Reranker")
ranked = model.retrieve(
query=query,
memories=memories,
evidence_reranker="v2",
top_k=10,
)
The repository layout is compatible with convmemory.EvidenceReranker.from_pretrained:
evidence_reranker_config.jsonMANIFEST.jsoncross_encoder/SentenceTransformers CrossEncoder checkpoint
What It Does
ConvMemory v2 preserves the exact ConvMemory v1 top-10 candidate set and only reorders that protected prefix using token-level query/memory evidence. It cannot recover a gold memory that v1 failed to retrieve into top-10.
Training
- Source experiment:
experiments/v361_top10_evidence_reranker.py - Seed: 7
- Base model:
cross-encoder/ms-marco-MiniLM-L-6-v2 - Training target: gold-only listwise retrieval cross-entropy
- Teacher weight: 0.0
- Candidate pool: ConvMemory v1 top-10 from dense MPNet top-500
Headline Evaluation
Canonical v361 5-seed headline, reported in the ConvMemory repository:
- ConvMemory v1 FULL MRR: 0.5824
- ConvMemory v2 FULL MRR: 0.6560
- Delta: +0.0734, paired bootstrap 95% CI [+0.0645, +0.0827]
The v364 load-bearing ablation retrained the same full-text arm in an ablation harness and obtained FULL MRR 0.6677. Text perturbations collapsed:
- no memory text: 0.2966
- random other-query text: 0.2506
- shuffled memory text: 0.2731
- scalar only: 0.5792
Anti-Leak Discipline
The public inference API rejects gold-defining or post-hoc fields such as:
gold, gold_ids, is_current, is_latest, is_stale, stale, answer,
answer_text, ce_score, mxbai_score, teacher_score, gpt_label,
entity_id, and slot_id.
Inference inputs are query text, candidate id/text, optional candidate position or time metadata, and the protected v1 top-10 candidate set.
Limitations
- LoCoMo-specific fine-tuning; validate or retrain before using cross-domain.
- Recall-preserving over v1 top-10, not a replacement for candidate generation.
- Not a full top-500 cross-encoder. It is a bounded precision stage after v1.
from sentence_transformers import CrossEncoder model = CrossEncoder("Purdy0228/ConvMemory-v2-Evidence-Reranker") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores)