Instructions to use KaLM-Embedding/KaLM-Reranker-V1-Large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use KaLM-Embedding/KaLM-Reranker-V1-Large with sentence-transformers:
from sentence_transformers import CrossEncoder model = CrossEncoder("KaLM-Embedding/KaLM-Reranker-V1-Large") query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) - Transformers
How to use KaLM-Embedding/KaLM-Reranker-V1-Large with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("KaLM-Embedding/KaLM-Reranker-V1-Large") model = AutoModelForMultimodalLM.from_pretrained("KaLM-Embedding/KaLM-Reranker-V1-Large", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add sentence-transformers CrossEncoder support
Sentence Transformers CrossEncoder support
Summary
This PR adds model-side support for loading the reranker through the modular
Sentence Transformers CrossEncoder API:
import torch
from sentence_transformers import CrossEncoder
model = CrossEncoder(
"KaLM-Embedding/KaLM-Reranker-V1-Large",
trust_remote_code=True,
device="cuda",
model_kwargs={"dtype": torch.bfloat16, "chunk_size": 4},
)
scores = model.predict([(query, document)])
rankings = model.rank(query, documents)
The document is encoded on the encoder side, while the task instruction and
query remain on the decoder side. The default Sigmoid activation returnsP(yes); passing activation_fn=torch.nn.Identity() returnsyes_logit - no_logit.
chunk_size=N performs mask-aware mean pooling over each consecutive group ofN encoder token states. Set chunk_size=None to disable encoder compression.
Files
The PR changes only these six adapter/configuration files:
kalm_reranker.pykalm_reranker_utils.pykalm_cross_encoder.pykalm_cross_encoder_config.jsonmodules.jsonconfig_sentence_transformers.json
It does not modify or upload the repository README, model weights, tokenizer
artifacts, generation configuration, or Transformers config.json.
Compatibility
sentence-transformers>=5.6,<6transformers>=5.3,<6- PyTorch inference backend only
The adapter code is shared with the validatedKaLM-Embedding/KaLM-Reranker-V1-Nano CrossEncoder integration. Small and
Large retain their existing model-specific Transformers configurations and
weights.
KaLM-Reranker-V1-Large validation
The six files in this PR were verified byte-for-byte against the locally tested
adapter files. The repository README at the PR revision is unchanged frommain.
H100 BF16 numerical smoke test
| Item | Result |
|---|---|
| sentence-transformers | 5.6.0 |
| transformers | 5.3.0 |
| torch | 2.6.0+cu126 |
| encoder chunk size | 4 |
| batch size | 2 |
| tolerance | rtol=1e-3, atol=2e-3 |
| max absolute error | 2.9103830456733704e-11 |
| ranking corpus IDs | [0, 1] |
Reference KaLMReranker scores and CrossEncoder scores agree within the BF16
tolerance for the five mixed English, Chinese, empty-query, and empty-document
inputs. The existing Large config.json, tokenizer, and model weights were
used without modification.
Immutable local checkpoint hashes observed during validation:
config.json:cb9cd9526c0669c3397e0e332215f9b2b642edca075dea43819002ce6debdfe2model.safetensors:94c59b9ce11b0f36c456419a3e43c9ca9c3a9ab4c86c1f97e51fdb896317e002tokenizer.json:f5b325224482ec441ec5fbe2a5ac08c3758e0f9605f6e54368e31f736fcfb01d