README.md

---
license: mit
language:
- en
base_model:
- Qwen/Qwen3-0.6B-Base
library_name: transformers
pipeline_tag: text-classification
tags:
- retrieval
- document-retrieval
- information-retrieval
- routing
- RAG
- query-routing
- late-interaction
- lora
- peft
datasets:
- emrekuruu/FinReport
- emrekuruu/FinSlides
- emrekuruu/FinQA
- emrekuruu/ConvFinQA
- emrekuruu/VQAonBD
- emrekuruu/TATDQA
- emrekuruu/ArxivQA
- emrekuruu/Wiki-ss
- emrekuruu/MP-DocVQA
- emrekuruu/SciQAG
- emrekuruu/DUDE
metrics:
- ndcg
---

# RetrievalRouter (λ=0.5)

Official checkpoint from **RetrievalRouter: Joint Modality and Architecture Selection for
Document Retrieval** (EMNLP 2026). Given only the **query text**, RetrievalRouter predicts
*which retrieval pipeline* — across **modality** (text vs. multimodal) and **architecture**
(lexical, dense, or late-interaction rerank) — to run for that query.

- 📄 Paper: https://arxiv.org/pdf/2608.25625
- 💻 Code: https://github.com/emrekuruu/retrieval-router
- 🤗 Collection (all checkpoints + datasets): https://huggingface.co/collections/emrekuruu/retrieval-router

## Motivation

Retrieval pipelines differ in **modality** (search over text, or over page images) and
**architecture** (cheap dense search, or expensive late-interaction). The accurate ones are
slow; the fast ones miss evidence on hard documents. And which one fails depends on the query —
a text pipeline can't answer "what's the red curve in Figure 3?", but a multimodal one is
overkill for a plain factoid. Across 11 benchmarks, **no single pipeline wins on everything**.
RetrievalRouter picks the cheapest pipeline that can still answer each query, so easy queries
stay fast and hard ones still get the heavy pipeline.

## What this model is for

This is a **router, not a retriever**. It takes a query and predicts which of five retrieval
pipelines to run — in about 15 ms, before any search happens. You then run the chosen pipeline
to fetch documents.

Use it when you keep several retrieval setups over the same corpus and want to run the expensive
ones only when they help. Pick the checkpoint by **λ**: `0.0` for best accuracy, `1.0` for best
speed, in between to trade off.

It doesn't rank or read documents itself, and assumes your indices already exist. Trained on
English financial, scientific, and open-domain documents; other domains and languages are
untested.

## This checkpoint

**Trained with λ=0.5** — a **balanced** objective (λ=0.5), trading a controlled amount of accuracy for lower latency between the quality-only (λ=0.0) and latency-only (λ=1.0) endpoints.

| Checkpoint | λ | Objective |
|---|---|---|
| [`RetrievalRouter-lambda-l00`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l00) | 0.0 | Accuracy only |
| [`RetrievalRouter-lambda-l10`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l10) | 0.1 | Accuracy-leaning |
| [`RetrievalRouter-lambda-l30`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l30) | 0.3 | Balanced |
| [`RetrievalRouter-lambda-l50`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l50) | 0.5 | Balanced |
| [`RetrievalRouter-lambda-l70`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l70) | 0.7 | Latency-leaning |
| [`RetrievalRouter-lambda-l100`](https://huggingface.co/emrekuruu/RetrievalRouter-lambda-l100) | 1.0 | Latency only |

## Routing arms

| Index | Arm (config name) | Paper name | Modality | Architecture |
|---|---|---|---|---|
| 0 | `MULTIMODAL_RERANK` | MM-Rerank | Multimodal | Dense → late-interaction rerank |
| 1 | `MULTIMODAL-SINGLE` | MM-Dense | Multimodal | Single-vector dense |
| 2 | `TEXT_RERANK` | Text-Rerank | Text | Dense → late-interaction rerank |
| 3 | `TEXT-SINGLE` | Text-Dense | Text | Single-vector dense |
| 4 | `BM25` | BM25 | Text | Lexical |

The action space is these five arms. Two further pipelines evaluated in the paper (Text-Late,
MM-Late) are reported as static reference baselines but never routed to.

## Architecture

- **Encoder:** [Qwen/Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) with LoRA
  adapters on the attention and feed-forward projections (merged into these weights).
- **Pooling:** mean-pool over the final hidden states → a 1024-d query representation.
- **Head:** a single linear layer → logits over the five arms; softmax gives the routing policy.
- Custom modeling code ships in the repo and loads via `trust_remote_code=True`.

## Usage

```python
import torch
from transformers import AutoModel, AutoTokenizer

repo = "emrekuruu/RetrievalRouter-lambda-l50"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()

inputs = tokenizer("In figure 3, what does the red dashed curve represent?",
                   return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    logits = model(**inputs)["logits"]              # shape [1, 5]
arm = model.config.strategy_names[logits.softmax(-1).argmax(-1).item()]
print(arm)   # e.g. "MULTIMODAL_RERANK" -> run that pipeline for this query
```

The router returns **which retrieval pipeline to run**, not documents. You then execute the
selected pipeline against your own indices.

## Training

Trained against **soft targets** from a per-query reward vector over the five arms, rather than a
single hard best-pipeline label (pipelines frequently tie on nDCG@5, and hard labels inject
noise). The reward combines accuracy and efficiency,

$$ r_i(q) = (1-\lambda)\, s_i(q) + \lambda\,(1 - \ell_i(q)), $$

where $s_i(q)$ is the arm's nDCG@5 and $\ell_i(q)$ its per-query normalized latency. The reward
vector becomes a target distribution via a low-temperature softmax (τ=0.1), and the router
minimizes the KL divergence to it. **λ is the only knob** that differs across the checkpoints
above. Training data spans **85,103 queries across 11 benchmarks**.

## Results (headline)

Against the strongest static pipeline, RetrievalRouter is **+2.5% nDCG@5 and 12.4× faster**.
Against the prior adaptive strategy-selection baseline
([`emrekuruu/RetrievalRouter_Baseline`](https://huggingface.co/emrekuruu/RetrievalRouter_Baseline)), it achieves significantly
higher nDCG@5 in accuracy-oriented settings and matches or numerically beats it on both nDCG@5
and latency in latency-oriented settings. See the paper for full tables and significance tests.

## Citation

```bibtex
@misc{kuru2026retrievalrouterjointmodalityarchitecture,
      title={RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval}, 
      author={Emre Kuru and Mehmet Onur Keskin and Reza Farahbakhsh and Noel Crespi},
      year={2026},
      eprint={2608.25625},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2608.25625}, 
}
```