metadata
inference: false
datasets:
- answerdotai/MMARCO-japanese-32-scored-triplets
- unicamp-dl/mmarco
language:
- ja
pipeline_tag: sentence-similarity
tags:
- ColBERT
- multi-vector
- sentence-transformers
base_model:
- cl-tohoku/bert-base-japanese-v3
- bclavie/JaColBERT
license: mit
library_name: RAGatouille
Model weights for the JaColBERTv2.4 checkpoint, which is the pre-post-training version of JaColBERTv2.5, using an entirely overhauled training recipe and trained on just 40% of the data of JaColBERTv2.
This model largely outperforms all previous approaches, including JaColBERTV2 multilingual models such as BGE-M3, on all datasets.
This page will be updated with the full details and the model report in the next few days.
Sentence Transformers
As of Sentence Transformers v6.0.0, this model can also be loaded directly as a multi-vector (ColBERT-style late interaction) retriever via the MultiVectorEncoder:
pip install "sentence-transformers>=6.0.0" fugashi unidic-lite
from sentence_transformers import MultiVectorEncoder
model = MultiVectorEncoder("answerdotai/JaColBERTv2.4")
query = "日本で一番高い山は何ですか？"
documents = [
"富士山は日本で最も高い山で、標高は3776メートルです。",
"東京は日本の首都で、世界最大の都市圏の一つです。",
"北岳は南アルプスにある山で、日本で二番目に高い山です。",
"富士山は静岡県と山梨県にまたがる活火山です。",
]
query_embeddings = model.encode_query(query)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings[0].shape)
# torch.Size([32, 128]) torch.Size([21, 128])
# MaxSim late-interaction scoring (higher is more relevant)
scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[30.6411, 28.8692, 29.7601, 28.9441]], device='cuda:0')
@misc{clavié2024jacolbertv25optimisingmultivectorretrievers,
title={JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create State-of-the-Art Japanese Retrievers with Constrained Resources},
author={Benjamin Clavié},
year={2024},
eprint={2407.20750},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2407.20750},
}