| | --- |
| | language: multilingual |
| | datasets: |
| | - NQ |
| | - Trivia |
| | - SQuAD |
| | - MLQA |
| | - DRCD |
| | --- |
| | |
| | # dpr-ctx_encoder-bert-base-multilingual |
| | |
| | ## Description |
| | |
| | Multilingual DPR Model base on bert-base-multilingual-cased. |
| | [DPR model](https://arxiv.org/abs/2004.04906) |
| | [DPR repo](https://github.com/facebookresearch/DPR) |
| | |
| | ## Data |
| | 1. [NQ](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py) |
| | 2. [Trivia](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py) |
| | 3. [SQuAD](https://github.com/facebookresearch/DPR/blob/master/data/download_data.py) |
| | 4. [DRCD*](https://github.com/DRCKnowledgeTeam/DRCD) |
| | 5. [MLQA*](https://github.com/facebookresearch/MLQA) |
| | |
| | `question pairs for train`： 644,217 |
| | `question pairs for dev`： 73,710 |
| | |
| | *DRCD and MLQA are converted using script from haystack [squad_to_dpr.py](https://github.com/deepset-ai/haystack/blob/master/haystack/retriever/squad_to_dpr.py) |
| | |
| | ## Training Script |
| | I use the script from [haystack](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial9_DPR_training.ipynb) |
| | |
| | ## Usage |
| | |
| | ```python |
| | from transformers import DPRContextEncoder, DPRContextEncoderTokenizer |
| | tokenizer = DPRContextEncoderTokenizer.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual') |
| | model = DPRContextEncoder.from_pretrained('voidful/dpr-ctx_encoder-bert-base-multilingual') |
| | input_ids = tokenizer("Hello, is my dog cute ?", return_tensors='pt')["input_ids"] |
| | embeddings = model(input_ids).pooler_output |
| | ``` |
| | |
| | Follow the tutorial from `haystack`: |
| | [Better Retrievers via "Dense Passage Retrieval"](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial6_Better_Retrieval_via_DPR.ipynb) |
| | ``` |
| | from haystack.retriever.dense import DensePassageRetriever |
| | retriever = DensePassageRetriever(document_store=document_store, |
| | query_embedding_model="voidful/dpr-question_encoder-bert-base-multilingual", |
| | passage_embedding_model="voidful/dpr-ctx_encoder-bert-base-multilingual", |
| | max_seq_len_query=64, |
| | max_seq_len_passage=256, |
| | batch_size=16, |
| | use_gpu=True, |
| | embed_title=True, |
| | use_fast_tokenizers=True) |
| | ``` |
| | |
