diff --git "a/index.html" "b/index.html" --- "a/index.html" +++ "b/index.html" @@ -1,19 +1,1143 @@ - - - - - - My static Space - - - -
-

Welcome to your static Space!

-

You can modify this app directly by editing index.html in the Files and versions tab.

-

- Also don't forget to check the - Spaces documentation. -

-
- - + + + + + +cheRAGh — Persian RAG Reranker Benchmark + + + + + + + +
+ +
+ +
+ + +
+
+ + + +
+
+
+ + + + + + +
+
+
+
+
+ + +
+
cheRAGh · چراغ · Reranker Benchmark
+ +

Persian RAG
Reranker Evaluation

+

بنچمارک RAG فارسی
ارزیابی مدل‌های Reranker

+ +
cheRAGh — Benchmarking Suite for Persian RAG Systems
+ + +
+
چراغِ جست‌وجو افروختیم در شامِ پرسش‌ها
+
که پیدا گردد از نورش، رهِ پاسخ زِ چالش‌ها
+
+ We lit the lamp of search in the night of questions,
+ so its light may reveal the path of answers from challenges. +
+
+ +

+ cheRAGh (چراغ) is a unified benchmarking suite for Persian Retrieval-Augmented Generation (RAG) systems, covering embedding models, rerankers, retrieval quality, tool calling, and large language model performance across diverse Persian-language datasets from General, Scientific, Education, Legal, and Religious domains. +

+

+ This report presents the evaluation results of reranker models in the cheRAGh benchmark. Each configuration pairs a base embedding model's retrieval with a reranker pass; performance is measured using Recall@5, MRR, and Δ MRR — the gain (or loss) the reranker contributes over base embedding-only retrieval. +

+ + + +
+
14
Configurations
+
5
Domains
+
0.870
Avg MRR
+
0.928
Avg Recall@5
+
+0.006
Avg Δ MRR
+
+
+ + +
+ + +
+ +
+
+
+ +
+ + +
+
+ 01 +
+
Benchmark Datasets
+ +
Five Persian-language domains — Legal, Religious, General, Scientific, and Education — each containing queries paired with a single ground-truth document. The base embedding model retrieves top-k candidates, the reranker re-orders them, and we compare ranking quality and retrieval coverage before and after reranking.
+ +
+
+
📚
Education
Official high-school textbook passages
🌐
General
Open-domain Q&A platform logs, everyday topics
⚖️
Legal
Legal statutes, court decisions & regulatory texts
🕌
Religious
Islamic texts, scholarly commentary & theological Q&A
🔬
Scientific
Peer-reviewed scientific paper abstracts
+
+ + +
+
+ 02 +
+
Leaderboard
+ +
+ Green ≥ 0.75 · + Orange 0.50–0.74 · + Red < 0.50 · + Bold green = best in column · + Δ MRR: green = improved, red = hurt by reranking +
+ +
+
+ +
+
+ + + + +
+ +
+
+ +
+ Sorted by Average MRR ↓ + +
+
+ + + + + + + + + + + + + + + + + + + + + +
# ↕Reranker ↕ArchitectureParams ↕SizeMax Len ↕Base ModelEducation ↕General ↕Legal ↕Religious ↕Scientific ↕Average ↕
MRR ↕Recall@5 ↕Δ MRR ↕MRR ↕Recall@5 ↕Δ MRR ↕MRR ↕Recall@5 ↕Δ MRR ↕MRR ↕Recall@5 ↕Δ MRR ↕MRR ↕Recall@5 ↕Δ MRR ↕MRR ↕Recall@5 ↕Δ MRR ↕
+
+
+ + + +
+
+ 03 +
+
Visualisations
+ +
+
+
+
+
MRR by Dataset
+ +
Mean Reciprocal Rank
+
+
+
+
Recall@5 by Dataset
+ +
Fraction of queries where ground-truth is in top-k
+ +
+
+
+
+
+
Δ MRR by Dataset
+ +
Reranker lift over base embedding retrieval
+ +
+
+
+
Radar — MRR
+
Multi-domain strength profile
+ +
+
+
+
+
+
Radar — Recall@5
+
Multi-domain coverage profile
+ +
+
+
+
+
+
+
Performance Heatmap
+ +
All configurations × datasets — scroll to explore · hover for value
+ +
+
+ + + +
+
+
+
+ Worst +
+ Best +
+
+
+ + +
+
+ 04 +
+
About the Metrics
+ +
+
+
+
+
MRR — Mean Reciprocal Rank
+
Averages 1/rank across all queries. Rank 1 every time → 1.0; rank 5 every time → 0.20. Rewards getting the right answer to the top.
+ +
+
+
Recall@5
+
Fraction of queries where the correct document appears in the top-k results.
+ +
+
+
Δ MRR — Reranker Lift
+
MRR after reranking minus MRR of the base embedding-only retrieval. Positive means the reranker improved ranking quality; negative means it hurt it.
+ +
+
+
+ About cheRAGh + +
+
+ cheRAGh (چراغ) is a unified benchmarking suite for Persian RAG systems, + covering embeddings, rerankers, retrieval quality, tool calling, and end-to-end pipeline performance. +
+ + +
+
+
+ +
+
cheRAGh · HuggingFace · 2026-07-25 09:12
+
+ ● ≥ 0.75   + ● 0.50–0.74   + ● < 0.50   + ● best +
+
+
+ + + + + + \ No newline at end of file