Update README.md

README.md

@@ -92,6 +92,7 @@ On LMEB, reranking models demonstrate a clear advantage, with even the 0.27B Nan
 ![lmeb_emb](./assets/lmeb_emb.jpg)
 
 # Usage
+## Using transformers
 ```python
 import argparse
 from typing import Optional
@@ -192,17 +193,121 @@ def main() -> None:
     pairs = [(query, document) for document in documents]
     print("scores:", reranker.predict(pairs, instruction=instruction))
     print("rankings:", reranker.rank(query, documents, instruction=instruction))
-
+    
 
 if __name__ == "__main__":
     main()
 
 '''
-scores: [0.9998464584350586, 6.893610020597407e-07]
-rankings: [{'corpus_id': 0, 'score': 0.9998464584350586}, {'corpus_id': 1, 'score': 6.893610020597407e-07}]
+scores: [0.9998205304145813, 4.7850949158601e-06]
+rankings: [{'corpus_id': 0, 'score': 0.9998205304145813}, {'corpus_id': 1, 'score': 4.7850949158601e-06}]
 '''
+
+```
+
+## Using vLLM
+An experimental single-GPU adapter is available for offline
+`LLM.classify()` reranking and optional FastAPI serving. It reuses the original
+checkpoint without adding or modifying model weights.
+
+The adapter has been validated with Python 3.12, vLLM 0.19.1, Transformers
+5.6.2 and CUDA BF16:
+
+```bash
+conda create -n kalm-vllm python=3.12 -y
+conda activate kalm-vllm
+pip install "vllm==0.19.1" "transformers==5.6.2"
+
+hf download KaLM-Embedding/KaLM-Reranker-V1-Large \
+  --local-dir ./KaLM-Reranker-V1-Large
+pip install ./KaLM-Reranker-V1-Large/vllm_support --no-deps
+export VLLM_PLUGINS=kalm_t5gemma2
+```
+
+Offline Python:
+
+```python
+from kalm_t5gemma2_vllm_plugin import KaLMVLLMReranker
+
+query = "What is the capital of China?"
+documents = [
+    "The capital of China is Beijing.",
+    "Gravity attracts bodies toward one another.",
+]
+
+with KaLMVLLMReranker(
+    "KaLM-Embedding/KaLM-Reranker-V1-Large",
+    query_max_length=512,
+    document_max_length=1024,
+    encoder_chunk_size=4,
+) as reranker:
+    print(reranker.rank(query, documents))
+```
+
+Offline CLI:
+
+```bash
+kalm-vllm-rerank --return-margin
 ```
 
+To deploy the online service, install the HTTP dependencies and keep the
+server running in the first terminal:
+
+```bash
+pip install "fastapi>=0.136,<0.137" "uvicorn>=0.46,<0.47"
+export CUDA_VISIBLE_DEVICES=0
+export VLLM_PLUGINS=kalm_t5gemma2
+
+kalm-vllm-serve \
+  --host 0.0.0.0 \
+  --port 8000 \
+  --model KaLM-Embedding/KaLM-Reranker-V1-Large \
+  --query-max-length 512 \
+  --document-max-length 1024 \
+  --encoder-chunk-size 4 \
+  --max-model-len 2048
+```
+
+In a second terminal, check the server:
+
+```bash
+conda activate kalm-vllm
+kalm-vllm-client --base-url http://127.0.0.1:8000 --health
+```
+
+Use `/rerank` for one query and a list of documents. Results are sorted by
+score:
+
+```bash
+kalm-vllm-client \
+  --base-url http://127.0.0.1:8000 \
+  --endpoint rerank \
+  --json-file ./KaLM-Reranker-V1-Large/vllm_support/examples/rerank_request.json \
+  --return-margin \
+  --top-k 10
+```
+
+Use `/score` to score a batch of independent query-document pairs. Results
+preserve the input order and optional IDs:
+
+```bash
+kalm-vllm-client \
+  --base-url http://127.0.0.1:8000 \
+  --endpoint score \
+  --json-file ./KaLM-Reranker-V1-Large/vllm_support/examples/score_request.json \
+  --return-margin
+```
+
+The default output is `P(yes)`. Set `return_margin=true` to also receive
+`yes_logit - no_logit`; the client flag `--return-margin` applies the same
+setting to a JSON file request. The supported encoder chunk sizes are
+`1, 2, 4, 8, 16, 32`, with `4` as the default.
+
+This adapter uses vLLM's plugin, scheduling and pooling interfaces while the
+T5Gemma2 semantic forward still runs through Transformers. It is not vLLM's
+native HTTP `/score` implementation or a complete vLLM-native kernel port.
+See [the complete installation, API and troubleshooting guide](./vllm_support/README.md).
+
 # Acknowledgements
 We sincerely thank `jina-reranker-v3` and `Qwen3-Reranker` for their valuable inspiration and contributions to the reranking community, from which we have learned a lot.