Feature Extraction
Transformers
Safetensors
sentence-transformers
minicpm
mteb
custom_code
Eval Results (legacy)
Instructions to use openbmb/MiniCPM-Embedding-Light with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM-Embedding-Light with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="openbmb/MiniCPM-Embedding-Light", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("openbmb/MiniCPM-Embedding-Light", trust_remote_code=True, dtype="auto") - sentence-transformers
How to use openbmb/MiniCPM-Embedding-Light with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("openbmb/MiniCPM-Embedding-Light", trust_remote_code=True) sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
metadata
model-index:
- name: no_model_name_available
results:
- dataset:
config: default
name: MTEB ArguAna (default)
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
split: test
type: mteb/arguana
metrics:
- type: main_score
value: 40.849000000000004
- type: map_at_1
value: 18.492
- type: map_at_10
value: 32.5
- type: map_at_100
value: 33.94
- type: map_at_1000
value: 33.955
- type: map_at_20
value: 33.58
- type: map_at_3
value: 27.500999999999998
- type: map_at_5
value: 30.279
- type: mrr_at_1
value: 18.776671408250355
- type: mrr_at_10
value: 32.60919754340802
- type: mrr_at_100
value: 34.042399188681074
- type: mrr_at_1000
value: 34.05821738840704
- type: mrr_at_20
value: 33.68314031545244
- type: mrr_at_3
value: 27.64343290659076
- type: mrr_at_5
value: 30.395922238027417
- type: nauc_map_at_1000_diff1
value: 1.8241584818848398
- type: nauc_map_at_1000_max
value: -3.009641035235681
- type: nauc_map_at_1000_std
value: -8.060365995660506
- type: nauc_map_at_100_diff1
value: 1.8458972346704023
- type: nauc_map_at_100_max
value: -2.972244792103319
- type: nauc_map_at_100_std
value: -8.03077807985782
- type: nauc_map_at_10_diff1
value: 1.70335000622157
- type: nauc_map_at_10_max
value: -2.9095471140113967
- type: nauc_map_at_10_std
value: -8.133921904552842
- type: nauc_map_at_1_diff1
value: 0.35369074889923846
- type: nauc_map_at_1_max
value: -8.485386718977434
- type: nauc_map_at_1_std
value: -11.609625990143924
- type: nauc_map_at_20_diff1
value: 1.6973541863717017
- type: nauc_map_at_20_max
value: -2.953190015586193
- type: nauc_map_at_20_std
value: -7.995561738947742
- type: nauc_map_at_3_diff1
value: 1.7643424351370773
- type: nauc_map_at_3_max
value: -5.015939381941595
- type: nauc_map_at_3_std
value: -9.460406898813288
- type: nauc_map_at_5_diff1
value: 2.1852763786361877
- type: nauc_map_at_5_max
value: -3.3865686219815463
- type: nauc_map_at_5_std
value: -8.742380012175492
- type: nauc_mrr_at_1000_diff1
value: 0.9406570687762197
- type: nauc_mrr_at_1000_max
value: -3.3634639176793093
- type: nauc_mrr_at_1000_std
value: -7.966276102315341
- type: nauc_mrr_at_100_diff1
value: 0.9627566529735759
- type: nauc_mrr_at_100_max
value: -3.3260791808487054
- type: nauc_mrr_at_100_std
value: -7.936900094905271
- type: nauc_mrr_at_10_diff1
value: 0.843267834551808
- type: nauc_mrr_at_10_max
value: -3.253260909530057
- type: nauc_mrr_at_10_std
value: -8.037042481229044
- type: nauc_mrr_at_1_diff1
value: -0.6844692690724743
- type: nauc_mrr_at_1_max
value: -8.511581129839898
- type: nauc_mrr_at_1_std
value: -10.923948005737325
- type: nauc_mrr_at_20_diff1
value: 0.8268205408842618
- type: nauc_mrr_at_20_max
value: -3.3011156704709257
- type: nauc_mrr_at_20_std
value: -7.901207586824542
- type: nauc_mrr_at_3_diff1
value: 1.0652496824313973
- type: nauc_mrr_at_3_max
value: -5.089357526316275
- type: nauc_mrr_at_3_std
value: -9.106877058656876
- type: nauc_mrr_at_5_diff1
value: 1.2998925500445453
- type: nauc_mrr_at_5_max
value: -3.713356146205814
- type: nauc_mrr_at_5_std
value: -8.609369461302613
- type: nauc_ndcg_at_1000_diff1
value: 2.322543696215367
- type: nauc_ndcg_at_1000_max
value: -1.075915240644691
- type: nauc_ndcg_at_1000_std
value: -6.400452085986812
- type: nauc_ndcg_at_100_diff1
value: 2.8404474222636926
- type: nauc_ndcg_at_100_max
value: -0.006028038831444503
- type: nauc_ndcg_at_100_std
value: -5.44699869879777
- type: nauc_ndcg_at_10_diff1
value: 1.7036471706518854
- type: nauc_ndcg_at_10_max
value: -0.14310599447899844
- type: nauc_ndcg_at_10_std
value: -6.217260401795631
- type: nauc_ndcg_at_1_diff1
value: 0.35369074889923846
- type: nauc_ndcg_at_1_max
value: -8.485386718977434
- type: nauc_ndcg_at_1_std
value: -11.609625990143924
- type: nauc_ndcg_at_20_diff1
value: 1.8564436158041036
- type: nauc_ndcg_at_20_max
value: 0.03537429283768104
- type: nauc_ndcg_at_20_std
value: -5.404784944281853
- type: nauc_ndcg_at_3_diff1
value: 2.140572253272633
- type: nauc_ndcg_at_3_max
value: -4.156002601722447
- type: nauc_ndcg_at_3_std
value: -8.940451632941194
- type: nauc_ndcg_at_5_diff1
value: 2.881440057342096
- type: nauc_ndcg_at_5_max
value: -1.3326095632764534
- type: nauc_ndcg_at_5_std
value: -7.694109014955247
- type: nauc_precision_at_1000_diff1
value: 28.555225089163237
- type: nauc_precision_at_1000_max
value: 57.32771738769801
- type: nauc_precision_at_1000_std
value: 58.3014480406798
- type: nauc_precision_at_100_diff1
value: 29.605300659573725
- type: nauc_precision_at_100_max
value: 59.138750768096884
- type: nauc_precision_at_100_std
value: 51.81829624122998
- type: nauc_precision_at_10_diff1
value: 1.2281902931429507
- type: nauc_precision_at_10_max
value: 9.547659655676522
- type: nauc_precision_at_10_std
value: 0.6831783299051362
- type: nauc_precision_at_1_diff1
value: 0.35369074889923846
- type: nauc_precision_at_1_max
value: -8.485386718977434
- type: nauc_precision_at_1_std
value: -11.609625990143924
- type: nauc_precision_at_20_diff1
value: 2.518025244990189
- type: nauc_precision_at_20_max
value: 17.604774216430066
- type: nauc_precision_at_20_std
value: 11.015897492994437
- type: nauc_precision_at_3_diff1
value: 3.085929777417008
- type: nauc_precision_at_3_max
value: -2.0135956625516127
- type: nauc_precision_at_3_std
value: -7.648912795110527
- type: nauc_precision_at_5_diff1
value: 4.789199864044247
- type: nauc_precision_at_5_max
value: 4.340341529819853
- type: nauc_precision_at_5_std
value: -4.829450482280661
- type: nauc_recall_at_1000_diff1
value: 28.555225089162
- type: nauc_recall_at_1000_max
value: 57.32771738769622
- type: nauc_recall_at_1000_std
value: 58.301448040679325
- type: nauc_recall_at_100_diff1
value: 29.60530065957345
- type: nauc_recall_at_100_max
value: 59.13875076809638
- type: nauc_recall_at_100_std
value: 51.818296241229376
- type: nauc_recall_at_10_diff1
value: 1.2281902931429811
- type: nauc_recall_at_10_max
value: 9.547659655676451
- type: nauc_recall_at_10_std
value: 0.6831783299050641
- type: nauc_recall_at_1_diff1
value: 0.35369074889923846
- type: nauc_recall_at_1_max
value: -8.485386718977434
- type: nauc_recall_at_1_std
value: -11.609625990143924
- type: nauc_recall_at_20_diff1
value: 2.518025244990186
- type: nauc_recall_at_20_max
value: 17.60477421643011
- type: nauc_recall_at_20_std
value: 11.015897492994425
- type: nauc_recall_at_3_diff1
value: 3.08592977741699
- type: nauc_recall_at_3_max
value: -2.0135956625516034
- type: nauc_recall_at_3_std
value: -7.648912795110507
- type: nauc_recall_at_5_diff1
value: 4.78919986404431
- type: nauc_recall_at_5_max
value: 4.3403415298198835
- type: nauc_recall_at_5_std
value: -4.829450482280615
- type: ndcg_at_1
value: 18.492
- type: ndcg_at_10
value: 40.849000000000004
- type: ndcg_at_100
value: 47.115
- type: ndcg_at_1000
value: 47.538000000000004
- type: ndcg_at_20
value: 44.727
- type: ndcg_at_3
value: 30.514999999999997
- type: ndcg_at_5
value: 35.507
- type: precision_at_1
value: 18.492
- type: precision_at_10
value: 6.7780000000000005
- type: precision_at_100
value: 0.954
- type: precision_at_1000
value: 0.099
- type: precision_at_20
value: 4.15
- type: precision_at_3
value: 13.086999999999998
- type: precision_at_5
value: 10.27
- type: recall_at_1
value: 18.492
- type: recall_at_10
value: 67.781
- type: recall_at_100
value: 95.448
- type: recall_at_1000
value: 98.791
- type: recall_at_20
value: 83.001
- type: recall_at_3
value: 39.26
- type: recall_at_5
value: 51.351
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackAndroidRetrieval (default)
revision: f46a197baaae43b4f621051089b82a364682dfeb
split: test
type: mteb/cqadupstack-android
metrics:
- type: main_score
value: 35.042
- type: map_at_1
value: 24.268
- type: map_at_10
value: 30.802000000000003
- type: map_at_100
value: 31.71
- type: map_at_1000
value: 31.828
- type: map_at_20
value: 31.305
- type: map_at_3
value: 28.701
- type: map_at_5
value: 29.749
- type: mrr_at_1
value: 29.756795422031473
- type: mrr_at_10
value: 35.65331425846446
- type: mrr_at_100
value: 36.34176813536559
- type: mrr_at_1000
value: 36.41234016532853
- type: mrr_at_20
value: 36.05978921961065
- type: mrr_at_3
value: 33.78636146876491
- type: mrr_at_5
value: 34.7377205531712
- type: nauc_map_at_1000_diff1
value: 50.27106929265
- type: nauc_map_at_1000_max
value: 43.135938626038886
- type: nauc_map_at_1000_std
value: 6.979578889406475
- type: nauc_map_at_100_diff1
value: 50.27717846143314
- type: nauc_map_at_100_max
value: 43.112426840531896
- type: nauc_map_at_100_std
value: 6.913838872983614
- type: nauc_map_at_10_diff1
value: 50.34272889874193
- type: nauc_map_at_10_max
value: 42.893739814095824
- type: nauc_map_at_10_std
value: 6.700454686853102
- type: nauc_map_at_1_diff1
value: 54.83912639955801
- type: nauc_map_at_1_max
value: 42.50639531768006
- type: nauc_map_at_1_std
value: 7.063585243381144
- type: nauc_map_at_20_diff1
value: 50.232349871503324
- type: nauc_map_at_20_max
value: 42.937078152890685
- type: nauc_map_at_20_std
value: 6.605312686330819
- type: nauc_map_at_3_diff1
value: 50.679087408987364
- type: nauc_map_at_3_max
value: 42.828233344110735
- type: nauc_map_at_3_std
value: 6.305755418874708
- type: nauc_map_at_5_diff1
value: 50.619556564982084
- type: nauc_map_at_5_max
value: 42.51573550587868
- type: nauc_map_at_5_std
value: 6.295959119532668
- type: nauc_mrr_at_1000_diff1
value: 50.89120084540395
- type: nauc_mrr_at_1000_max
value: 44.01341349287708
- type: nauc_mrr_at_1000_std
value: 8.684194220017558
- type: nauc_mrr_at_100_diff1
value: 50.90586330977123
- type: nauc_mrr_at_100_max
value: 44.01184920133466
- type: nauc_mrr_at_100_std
value: 8.6528905753814
- type: nauc_mrr_at_10_diff1
value: 51.08330854155516
- type: nauc_mrr_at_10_max
value: 44.16083674113598
- type: nauc_mrr_at_10_std
value: 8.672749143946042
- type: nauc_mrr_at_1_diff1
value: 55.379664251629
- type: nauc_mrr_at_1_max
value: 45.5763056221476
- type: nauc_mrr_at_1_std
value: 9.372699665662898
- type: nauc_mrr_at_20_diff1
value: 50.97197213680204
- type: nauc_mrr_at_20_max
value: 44.00145045510066
- type: nauc_mrr_at_20_std
value: 8.525331492315571
- type: nauc_mrr_at_3_diff1
value: 51.67410864524864
- type: nauc_mrr_at_3_max
value: 44.73092939542603
- type: nauc_mrr_at_3_std
value: 8.722441142976763
- type: nauc_mrr_at_5_diff1
value: 51.72383213008365
- type: nauc_mrr_at_5_max
value: 44.34672865435124
- type: nauc_mrr_at_5_std
value: 8.587250675881378
- type: nauc_ndcg_at_1000_diff1
value: 47.895888150498216
- type: nauc_ndcg_at_1000_max
value: 43.0370824420739
- type: nauc_ndcg_at_1000_std
value: 9.430079898002509
- type: nauc_ndcg_at_100_diff1
value: 48.12703703246235
- type: nauc_ndcg_at_100_max
value: 42.69414925369428
- type: nauc_ndcg_at_100_std
value: 7.907032123880726
- type: nauc_ndcg_at_10_diff1
value: 48.73220685116636
- type: nauc_ndcg_at_10_max
value: 42.9216872694305
- type: nauc_ndcg_at_10_std
value: 7.416368391386961
- type: nauc_ndcg_at_1_diff1
value: 55.379664251629
- type: nauc_ndcg_at_1_max
value: 45.5763056221476
- type: nauc_ndcg_at_1_std
value: 9.372699665662898
- type: nauc_ndcg_at_20_diff1
value: 48.14164237845679
- type: nauc_ndcg_at_20_max
value: 42.37776004540794
- type: nauc_ndcg_at_20_std
value: 6.661363737722851
- type: nauc_ndcg_at_3_diff1
value: 50.18742327204456
- type: nauc_ndcg_at_3_max
value: 44.25183514075081
- type: nauc_ndcg_at_3_std
value: 7.69785946841004
- type: nauc_ndcg_at_5_diff1
value: 49.72454972713352
- type: nauc_ndcg_at_5_max
value: 42.7925586511498
- type: nauc_ndcg_at_5_std
value: 6.940750282599943
- type: nauc_precision_at_1000_diff1
value: -1.716736142092752
- type: nauc_precision_at_1000_max
value: 9.356948862118577
- type: nauc_precision_at_1000_std
value: 16.694278195680575
- type: nauc_precision_at_100_diff1
value: 18.027040933632236
- type: nauc_precision_at_100_max
value: 25.167213490179357
- type: nauc_precision_at_100_std
value: 14.286766614163938
- type: nauc_precision_at_10_diff1
value: 33.908756901785985
- type: nauc_precision_at_10_max
value: 38.416502210707186
- type: nauc_precision_at_10_std
value: 13.105833934415893
- type: nauc_precision_at_1_diff1
value: 55.379664251629
- type: nauc_precision_at_1_max
value: 45.5763056221476
- type: nauc_precision_at_1_std
value: 9.372699665662898
- type: nauc_precision_at_20_diff1
value: 28.905770827169654
- type: nauc_precision_at_20_max
value: 34.727512522066334
- type: nauc_precision_at_20_std
value: 11.55620431316267
- type: nauc_precision_at_3_diff1
value: 42.93685104710452
- type: nauc_precision_at_3_max
value: 43.5795143589386
- type: nauc_precision_at_3_std
value: 9.49699580078321
- type: nauc_precision_at_5_diff1
value: 40.97125997308073
- type: nauc_precision_at_5_max
value: 41.42245252084514
- type: nauc_precision_at_5_std
value: 10.338095236912038
- type: nauc_recall_at_1000_diff1
value: 25.223365532755516
- type: nauc_recall_at_1000_max
value: 35.892090052954316
- type: nauc_recall_at_1000_std
value: 27.789895129443693
- type: nauc_recall_at_100_diff1
value: 34.81556616017488
- type: nauc_recall_at_100_max
value: 35.54471365446511
- type: nauc_recall_at_100_std
value: 8.85723605173314
- type: nauc_recall_at_10_diff1
value: 41.13579715967583
- type: nauc_recall_at_10_max
value: 38.51858224041396
- type: nauc_recall_at_10_std
value: 5.837427515276572
- type: nauc_recall_at_1_diff1
value: 54.83912639955801
- type: nauc_recall_at_1_max
value: 42.50639531768006
- type: nauc_recall_at_1_std
value: 7.063585243381144
- type: nauc_recall_at_20_diff1
value: 37.7252649482009
- type: nauc_recall_at_20_max
value: 35.9806766055433
- type: nauc_recall_at_20_std
value: 3.4231900712737025
- type: nauc_recall_at_3_diff1
value: 45.78506907885203
- type: nauc_recall_at_3_max
value: 41.15359226080764
- type: nauc_recall_at_3_std
value: 5.646787788216677
- type: nauc_recall_at_5_diff1
value: 44.729355622106645
- type: nauc_recall_at_5_max
value: 38.15303556670624
- type: nauc_recall_at_5_std
value: 4.411446149797608
- type: ndcg_at_1
value: 29.757
- type: ndcg_at_10
value: 35.042
- type: ndcg_at_100
value: 38.934000000000005
- type: ndcg_at_1000
value: 41.839
- type: ndcg_at_20
value: 36.544
- type: ndcg_at_3
value: 31.677
- type: ndcg_at_5
value: 32.926
- type: precision_at_1
value: 29.757
- type: precision_at_10
value: 6.18
- type: precision_at_100
value: 0.97
- type: precision_at_1000
value: 0.151
- type: precision_at_20
value: 3.605
- type: precision_at_3
value: 14.545
- type: precision_at_5
value: 10.043000000000001
- type: recall_at_1
value: 24.268
- type: recall_at_10
value: 42.897
- type: recall_at_100
value: 59.86900000000001
- type: recall_at_1000
value: 80.06299999999999
- type: recall_at_20
value: 48.555
- type: recall_at_3
value: 32.628
- type: recall_at_5
value: 36.525
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackEnglishRetrieval (default)
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
split: test
type: mteb/cqadupstack-english
metrics:
- type: main_score
value: 34.694
- type: map_at_1
value: 23.594
- type: map_at_10
value: 30.503999999999998
- type: map_at_100
value: 31.479000000000003
- type: map_at_1000
value: 31.606
- type: map_at_20
value: 31.025999999999996
- type: map_at_3
value: 28.688000000000002
- type: map_at_5
value: 29.744
- type: mrr_at_1
value: 29.554140127388536
- type: mrr_at_10
value: 35.92659993933879
- type: mrr_at_100
value: 36.58420519834615
- type: mrr_at_1000
value: 36.6435546754376
- type: mrr_at_20
value: 36.319116206832874
- type: mrr_at_3
value: 34.320594479830135
- type: mrr_at_5
value: 35.31104033970276
- type: nauc_map_at_1000_diff1
value: 54.50850619675715
- type: nauc_map_at_1000_max
value: 51.9706327719098
- type: nauc_map_at_1000_std
value: 8.337218581451756
- type: nauc_map_at_100_diff1
value: 54.56427906874124
- type: nauc_map_at_100_max
value: 51.9829483337371
- type: nauc_map_at_100_std
value: 8.288141086149269
- type: nauc_map_at_10_diff1
value: 55.073592238523275
- type: nauc_map_at_10_max
value: 52.22343980053664
- type: nauc_map_at_10_std
value: 8.043697808963204
- type: nauc_map_at_1_diff1
value: 59.86621051900818
- type: nauc_map_at_1_max
value: 50.04516021776138
- type: nauc_map_at_1_std
value: 5.827816350790341
- type: nauc_map_at_20_diff1
value: 54.794608821639414
- type: nauc_map_at_20_max
value: 52.078882356679365
- type: nauc_map_at_20_std
value: 8.092640916558945
- type: nauc_map_at_3_diff1
value: 55.87913345984871
- type: nauc_map_at_3_max
value: 52.25534686944724
- type: nauc_map_at_3_std
value: 7.45687862927487
- type: nauc_map_at_5_diff1
value: 55.37674613416643
- type: nauc_map_at_5_max
value: 52.32380378939199
- type: nauc_map_at_5_std
value: 8.008521085172447
- type: nauc_mrr_at_1000_diff1
value: 54.02228322670952
- type: nauc_mrr_at_1000_max
value: 53.685833475887804
- type: nauc_mrr_at_1000_std
value: 9.811694046835957
- type: nauc_mrr_at_100_diff1
value: 54.0198251104709
- type: nauc_mrr_at_100_max
value: 53.67903724249188
- type: nauc_mrr_at_100_std
value: 9.809076287739428
- type: nauc_mrr_at_10_diff1
value: 54.29375864779078
- type: nauc_mrr_at_10_max
value: 53.88648920775948
- type: nauc_mrr_at_10_std
value: 9.692969206949932
- type: nauc_mrr_at_1_diff1
value: 58.34757973069973
- type: nauc_mrr_at_1_max
value: 54.10981111531085
- type: nauc_mrr_at_1_std
value: 8.435745569168143
- type: nauc_mrr_at_20_diff1
value: 54.09149885113492
- type: nauc_mrr_at_20_max
value: 53.7252429683779
- type: nauc_mrr_at_20_std
value: 9.704109589554776
- type: nauc_mrr_at_3_diff1
value: 54.96714696529417
- type: nauc_mrr_at_3_max
value: 54.1223969071475
- type: nauc_mrr_at_3_std
value: 9.457399811310413
- type: nauc_mrr_at_5_diff1
value: 54.50251432270016
- type: nauc_mrr_at_5_max
value: 53.90651719065417
- type: nauc_mrr_at_5_std
value: 9.713550802739707
- type: nauc_ndcg_at_1000_diff1
value: 51.31565983669981
- type: nauc_ndcg_at_1000_max
value: 51.56533459138964
- type: nauc_ndcg_at_1000_std
value: 10.570401313447991
- type: nauc_ndcg_at_100_diff1
value: 51.503523680259434
- type: nauc_ndcg_at_100_max
value: 51.183287045185565
- type: nauc_ndcg_at_100_std
value: 9.908587658100721
- type: nauc_ndcg_at_10_diff1
value: 53.027727297749536
- type: nauc_ndcg_at_10_max
value: 52.18633021088797
- type: nauc_ndcg_at_10_std
value: 8.916658596147668
- type: nauc_ndcg_at_1_diff1
value: 58.34757973069973
- type: nauc_ndcg_at_1_max
value: 54.10981111531085
- type: nauc_ndcg_at_1_std
value: 8.435745569168143
- type: nauc_ndcg_at_20_diff1
value: 52.41659582379541
- type: nauc_ndcg_at_20_max
value: 51.71015438337976
- type: nauc_ndcg_at_20_std
value: 8.89908066016637
- type: nauc_ndcg_at_3_diff1
value: 53.65524010984417
- type: nauc_ndcg_at_3_max
value: 52.77730677162853
- type: nauc_ndcg_at_3_std
value: 8.734442828591241
- type: nauc_ndcg_at_5_diff1
value: 53.317774839239426
- type: nauc_ndcg_at_5_max
value: 52.37257522945775
- type: nauc_ndcg_at_5_std
value: 8.965526173742184
- type: nauc_precision_at_1000_diff1
value: -8.931148438903069
- type: nauc_precision_at_1000_max
value: 3.9841234834488177
- type: nauc_precision_at_1000_std
value: 10.833126796708637
- type: nauc_precision_at_100_diff1
value: 1.7322368541730724
- type: nauc_precision_at_100_max
value: 14.125858820069295
- type: nauc_precision_at_100_std
value: 11.552328836798782
- type: nauc_precision_at_10_diff1
value: 26.275350798297886
- type: nauc_precision_at_10_max
value: 39.18143990690451
- type: nauc_precision_at_10_std
value: 12.596419593485761
- type: nauc_precision_at_1_diff1
value: 58.34757973069973
- type: nauc_precision_at_1_max
value: 54.10981111531085
- type: nauc_precision_at_1_std
value: 8.435745569168143
- type: nauc_precision_at_20_diff1
value: 17.99651325874507
- type: nauc_precision_at_20_max
value: 31.018507068587738
- type: nauc_precision_at_20_std
value: 10.937503513046966
- type: nauc_precision_at_3_diff1
value: 40.20537287461531
- type: nauc_precision_at_3_max
value: 49.964606416920944
- type: nauc_precision_at_3_std
value: 11.877700369508975
- type: nauc_precision_at_5_diff1
value: 34.14161953881431
- type: nauc_precision_at_5_max
value: 46.25922060296257
- type: nauc_precision_at_5_std
value: 13.830816390982918
- type: nauc_recall_at_1000_diff1
value: 34.80722656503794
- type: nauc_recall_at_1000_max
value: 43.75668030978373
- type: nauc_recall_at_1000_std
value: 18.24364269385629
- type: nauc_recall_at_100_diff1
value: 38.54008758096123
- type: nauc_recall_at_100_max
value: 42.27687357480639
- type: nauc_recall_at_100_std
value: 12.866257637965282
- type: nauc_recall_at_10_diff1
value: 47.07862071194338
- type: nauc_recall_at_10_max
value: 48.41378203070431
- type: nauc_recall_at_10_std
value: 8.672134215209635
- type: nauc_recall_at_1_diff1
value: 59.86621051900818
- type: nauc_recall_at_1_max
value: 50.04516021776138
- type: nauc_recall_at_1_std
value: 5.827816350790341
- type: nauc_recall_at_20_diff1
value: 43.906182342755834
- type: nauc_recall_at_20_max
value: 46.092016425349904
- type: nauc_recall_at_20_std
value: 8.957854855647389
- type: nauc_recall_at_3_diff1
value: 50.789610919158434
- type: nauc_recall_at_3_max
value: 50.09843129373151
- type: nauc_recall_at_3_std
value: 7.779119287496343
- type: nauc_recall_at_5_diff1
value: 48.9221689680451
- type: nauc_recall_at_5_max
value: 48.937132920664325
- type: nauc_recall_at_5_std
value: 8.389357665338053
- type: ndcg_at_1
value: 29.554000000000002
- type: ndcg_at_10
value: 34.694
- type: ndcg_at_100
value: 38.607
- type: ndcg_at_1000
value: 41.031
- type: ndcg_at_20
value: 36.17
- type: ndcg_at_3
value: 32.168
- type: ndcg_at_5
value: 33.397
- type: precision_at_1
value: 29.554000000000002
- type: precision_at_10
value: 6.254999999999999
- type: precision_at_100
value: 1.068
- type: precision_at_1000
value: 0.154
- type: precision_at_20
value: 3.71
- type: precision_at_3
value: 15.329
- type: precision_at_5
value: 10.65
- type: recall_at_1
value: 23.594
- type: recall_at_10
value: 41.302
- type: recall_at_100
value: 57.906
- type: recall_at_1000
value: 73.99
- type: recall_at_20
value: 46.754
- type: recall_at_3
value: 33.254
- type: recall_at_5
value: 36.933
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGamingRetrieval (default)
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
split: test
type: mteb/cqadupstack-gaming
metrics:
- type: main_score
value: 45.028
- type: map_at_1
value: 31.915
- type: map_at_10
value: 40.522000000000006
- type: map_at_100
value: 41.382000000000005
- type: map_at_1000
value: 41.449999999999996
- type: map_at_20
value: 40.983000000000004
- type: map_at_3
value: 38.191
- type: map_at_5
value: 39.586
- type: mrr_at_1
value: 36.739811912225704
- type: mrr_at_10
value: 44.025177887246905
- type: mrr_at_100
value: 44.66382221892361
- type: mrr_at_1000
value: 44.710687367339204
- type: mrr_at_20
value: 44.387142873851595
- type: mrr_at_3
value: 42.08986415882968
- type: mrr_at_5
value: 43.2371995820272
- type: nauc_map_at_1000_diff1
value: 56.27862182703629
- type: nauc_map_at_1000_max
value: 50.107126351375854
- type: nauc_map_at_1000_std
value: 15.656754868200673
- type: nauc_map_at_100_diff1
value: 56.28573102728189
- type: nauc_map_at_100_max
value: 50.11064355133309
- type: nauc_map_at_100_std
value: 15.62294780583473
- type: nauc_map_at_10_diff1
value: 56.36912566686038
- type: nauc_map_at_10_max
value: 49.82712246732209
- type: nauc_map_at_10_std
value: 15.21268035237868
- type: nauc_map_at_1_diff1
value: 62.32681597104861
- type: nauc_map_at_1_max
value: 46.11111144800607
- type: nauc_map_at_1_std
value: 12.985042402836761
- type: nauc_map_at_20_diff1
value: 56.320353720183405
- type: nauc_map_at_20_max
value: 49.983756729628624
- type: nauc_map_at_20_std
value: 15.352618563526594
- type: nauc_map_at_3_diff1
value: 57.373326864190155
- type: nauc_map_at_3_max
value: 48.85610213553234
- type: nauc_map_at_3_std
value: 14.227290354373581
- type: nauc_map_at_5_diff1
value: 56.88900234457732
- type: nauc_map_at_5_max
value: 49.555112855322136
- type: nauc_map_at_5_std
value: 14.709857277428808
- type: nauc_mrr_at_1000_diff1
value: 56.090340648188366
- type: nauc_mrr_at_1000_max
value: 51.39996372541806
- type: nauc_mrr_at_1000_std
value: 16.019923666577753
- type: nauc_mrr_at_100_diff1
value: 56.083110575611386
- type: nauc_mrr_at_100_max
value: 51.40497573920969
- type: nauc_mrr_at_100_std
value: 16.004195899925076
- type: nauc_mrr_at_10_diff1
value: 56.09461877706852
- type: nauc_mrr_at_10_max
value: 51.267677674757515
- type: nauc_mrr_at_10_std
value: 15.840837492104232
- type: nauc_mrr_at_1_diff1
value: 61.55103690488107
- type: nauc_mrr_at_1_max
value: 49.941817920324475
- type: nauc_mrr_at_1_std
value: 14.995168020783323
- type: nauc_mrr_at_20_diff1
value: 56.08320477899063
- type: nauc_mrr_at_20_max
value: 51.36462184978103
- type: nauc_mrr_at_20_std
value: 15.874969148938655
- type: nauc_mrr_at_3_diff1
value: 57.06320707726257
- type: nauc_mrr_at_3_max
value: 51.01102923342208
- type: nauc_mrr_at_3_std
value: 15.391980997372984
- type: nauc_mrr_at_5_diff1
value: 56.63979096903498
- type: nauc_mrr_at_5_max
value: 51.26911175970512
- type: nauc_mrr_at_5_std
value: 15.489687246025424
- type: nauc_ndcg_at_1000_diff1
value: 53.58984801728004
- type: nauc_ndcg_at_1000_max
value: 51.74611650326185
- type: nauc_ndcg_at_1000_std
value: 18.261077541099084
- type: nauc_ndcg_at_100_diff1
value: 53.552895675287836
- type: nauc_ndcg_at_100_max
value: 51.86680482507793
- type: nauc_ndcg_at_100_std
value: 17.74920513508539
- type: nauc_ndcg_at_10_diff1
value: 53.91128717528939
- type: nauc_ndcg_at_10_max
value: 50.94123634232307
- type: nauc_ndcg_at_10_std
value: 16.178552032699166
- type: nauc_ndcg_at_1_diff1
value: 61.55103690488107
- type: nauc_ndcg_at_1_max
value: 49.941817920324475
- type: nauc_ndcg_at_1_std
value: 14.995168020783323
- type: nauc_ndcg_at_20_diff1
value: 53.72114596521803
- type: nauc_ndcg_at_20_max
value: 51.25100744970328
- type: nauc_ndcg_at_20_std
value: 16.329225958420494
- type: nauc_ndcg_at_3_diff1
value: 55.873468970955805
- type: nauc_ndcg_at_3_max
value: 49.83792697840682
- type: nauc_ndcg_at_3_std
value: 14.800025547991492
- type: nauc_ndcg_at_5_diff1
value: 55.244590535983065
- type: nauc_ndcg_at_5_max
value: 50.77060704577844
- type: nauc_ndcg_at_5_std
value: 15.333606597251311
- type: nauc_precision_at_1000_diff1
value: -2.1064382592680406
- type: nauc_precision_at_1000_max
value: 18.12791300721221
- type: nauc_precision_at_1000_std
value: 20.44672996587799
- type: nauc_precision_at_100_diff1
value: 9.722046784658827
- type: nauc_precision_at_100_max
value: 30.642705457608827
- type: nauc_precision_at_100_std
value: 22.020525764661404
- type: nauc_precision_at_10_diff1
value: 27.84006342301715
- type: nauc_precision_at_10_max
value: 43.5141465054166
- type: nauc_precision_at_10_std
value: 19.532164596593788
- type: nauc_precision_at_1_diff1
value: 61.55103690488107
- type: nauc_precision_at_1_max
value: 49.941817920324475
- type: nauc_precision_at_1_std
value: 14.995168020783323
- type: nauc_precision_at_20_diff1
value: 21.466984240449367
- type: nauc_precision_at_20_max
value: 39.74095492345742
- type: nauc_precision_at_20_std
value: 19.715479624658013
- type: nauc_precision_at_3_diff1
value: 41.588050878937544
- type: nauc_precision_at_3_max
value: 47.651039118304254
- type: nauc_precision_at_3_std
value: 15.928918353696412
- type: nauc_precision_at_5_diff1
value: 35.42959731150186
- type: nauc_precision_at_5_max
value: 46.40079493508327
- type: nauc_precision_at_5_std
value: 17.252010799453583
- type: nauc_recall_at_1000_diff1
value: 33.76196603236328
- type: nauc_recall_at_1000_max
value: 57.09289054088538
- type: nauc_recall_at_1000_std
value: 40.65879945206679
- type: nauc_recall_at_100_diff1
value: 39.565515468144305
- type: nauc_recall_at_100_max
value: 55.150856008837344
- type: nauc_recall_at_100_std
value: 27.090048238689025
- type: nauc_recall_at_10_diff1
value: 44.5937276757672
- type: nauc_recall_at_10_max
value: 50.560916580657725
- type: nauc_recall_at_10_std
value: 17.274039145018314
- type: nauc_recall_at_1_diff1
value: 62.32681597104861
- type: nauc_recall_at_1_max
value: 46.11111144800607
- type: nauc_recall_at_1_std
value: 12.985042402836761
- type: nauc_recall_at_20_diff1
value: 43.135661958475964
- type: nauc_recall_at_20_max
value: 51.32720993767131
- type: nauc_recall_at_20_std
value: 17.751145619620328
- type: nauc_recall_at_3_diff1
value: 51.253903709244206
- type: nauc_recall_at_3_max
value: 48.75306602942186
- type: nauc_recall_at_3_std
value: 14.1555348111707
- type: nauc_recall_at_5_diff1
value: 49.203081954513486
- type: nauc_recall_at_5_max
value: 50.46465274780778
- type: nauc_recall_at_5_std
value: 14.930739853068292
- type: ndcg_at_1
value: 36.74
- type: ndcg_at_10
value: 45.028
- type: ndcg_at_100
value: 48.872
- type: ndcg_at_1000
value: 50.553000000000004
- type: ndcg_at_20
value: 46.477000000000004
- type: ndcg_at_3
value: 41.024
- type: ndcg_at_5
value: 43.076
- type: precision_at_1
value: 36.74
- type: precision_at_10
value: 6.966
- type: precision_at_100
value: 0.9690000000000001
- type: precision_at_1000
value: 0.116
- type: precision_at_20
value: 3.9
- type: precision_at_3
value: 17.827
- type: precision_at_5
value: 12.176
- type: recall_at_1
value: 31.915
- type: recall_at_10
value: 55.125
- type: recall_at_100
value: 72.245
- type: recall_at_1000
value: 84.7
- type: recall_at_20
value: 60.55199999999999
- type: recall_at_3
value: 44.208
- type: recall_at_5
value: 49.279
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackGisRetrieval (default)
revision: 5003b3064772da1887988e05400cf3806fe491f2
split: test
type: mteb/cqadupstack-gis
metrics:
- type: main_score
value: 26.869
- type: map_at_1
value: 18.177
- type: map_at_10
value: 23.742
- type: map_at_100
value: 24.5
- type: map_at_1000
value: 24.596999999999998
- type: map_at_20
value: 24.115000000000002
- type: map_at_3
value: 22.256999999999998
- type: map_at_5
value: 23.093
- type: mrr_at_1
value: 19.661016949152543
- type: mrr_at_10
value: 25.375885570800822
- type: mrr_at_100
value: 26.118317661165392
- type: mrr_at_1000
value: 26.205933122139925
- type: mrr_at_20
value: 25.760881599825296
- type: mrr_at_3
value: 23.954802259887014
- type: mrr_at_5
value: 24.740112994350287
- type: nauc_map_at_1000_diff1
value: 43.91538215574489
- type: nauc_map_at_1000_max
value: 35.723591948633
- type: nauc_map_at_1000_std
value: 5.159156706467083
- type: nauc_map_at_100_diff1
value: 43.918111331867166
- type: nauc_map_at_100_max
value: 35.732249707572215
- type: nauc_map_at_100_std
value: 5.13339088499136
- type: nauc_map_at_10_diff1
value: 44.240971339067805
- type: nauc_map_at_10_max
value: 35.88110629607297
- type: nauc_map_at_10_std
value: 4.97891011883125
- type: nauc_map_at_1_diff1
value: 51.071763745176526
- type: nauc_map_at_1_max
value: 36.354013439554656
- type: nauc_map_at_1_std
value: 4.009623246187462
- type: nauc_map_at_20_diff1
value: 44.11106384006441
- type: nauc_map_at_20_max
value: 35.73522097161658
- type: nauc_map_at_20_std
value: 4.922649778568136
- type: nauc_map_at_3_diff1
value: 45.59230090767753
- type: nauc_map_at_3_max
value: 35.98323849432203
- type: nauc_map_at_3_std
value: 4.0881555020671145
- type: nauc_map_at_5_diff1
value: 44.90892815187525
- type: nauc_map_at_5_max
value: 36.08930265295397
- type: nauc_map_at_5_std
value: 4.778901969461804
- type: nauc_mrr_at_1000_diff1
value: 42.14524014011001
- type: nauc_mrr_at_1000_max
value: 36.164169080069236
- type: nauc_mrr_at_1000_std
value: 5.855559217369601
- type: nauc_mrr_at_100_diff1
value: 42.1411387906275
- type: nauc_mrr_at_100_max
value: 36.181873531339136
- type: nauc_mrr_at_100_std
value: 5.82404945394242
- type: nauc_mrr_at_10_diff1
value: 42.36398305226718
- type: nauc_mrr_at_10_max
value: 36.300239193756184
- type: nauc_mrr_at_10_std
value: 5.806333438431309
- type: nauc_mrr_at_1_diff1
value: 48.35468594232892
- type: nauc_mrr_at_1_max
value: 36.961573378969376
- type: nauc_mrr_at_1_std
value: 4.676168765221803
- type: nauc_mrr_at_20_diff1
value: 42.29977745460127
- type: nauc_mrr_at_20_max
value: 36.246164396778155
- type: nauc_mrr_at_20_std
value: 5.752225241570153
- type: nauc_mrr_at_3_diff1
value: 43.682033766308095
- type: nauc_mrr_at_3_max
value: 36.54761984785815
- type: nauc_mrr_at_3_std
value: 5.064386466886235
- type: nauc_mrr_at_5_diff1
value: 43.099309445019564
- type: nauc_mrr_at_5_max
value: 36.58856198590481
- type: nauc_mrr_at_5_std
value: 5.573063594553006
- type: nauc_ndcg_at_1000_diff1
value: 40.01688269619406
- type: nauc_ndcg_at_1000_max
value: 35.09587803542452
- type: nauc_ndcg_at_1000_std
value: 7.8300153260549425
- type: nauc_ndcg_at_100_diff1
value: 39.43500014960865
- type: nauc_ndcg_at_100_max
value: 35.006058224521816
- type: nauc_ndcg_at_100_std
value: 7.066507077867236
- type: nauc_ndcg_at_10_diff1
value: 41.15020558085733
- type: nauc_ndcg_at_10_max
value: 35.656022625409065
- type: nauc_ndcg_at_10_std
value: 6.152226178586941
- type: nauc_ndcg_at_1_diff1
value: 48.35468594232892
- type: nauc_ndcg_at_1_max
value: 36.961573378969376
- type: nauc_ndcg_at_1_std
value: 4.676168765221803
- type: nauc_ndcg_at_20_diff1
value: 40.77258392545088
- type: nauc_ndcg_at_20_max
value: 35.24647830668003
- type: nauc_ndcg_at_20_std
value: 5.928937453972604
- type: nauc_ndcg_at_3_diff1
value: 43.89011242026087
- type: nauc_ndcg_at_3_max
value: 36.37450968284067
- type: nauc_ndcg_at_3_std
value: 4.939095464964828
- type: nauc_ndcg_at_5_diff1
value: 42.811533700663794
- type: nauc_ndcg_at_5_max
value: 36.268311390109936
- type: nauc_ndcg_at_5_std
value: 5.812581090379655
- type: nauc_precision_at_1000_diff1
value: 10.645037512531824
- type: nauc_precision_at_1000_max
value: 25.700369403747942
- type: nauc_precision_at_1000_std
value: 21.03999886177142
- type: nauc_precision_at_100_diff1
value: 19.629282394242015
- type: nauc_precision_at_100_max
value: 31.764564604957553
- type: nauc_precision_at_100_std
value: 15.728081077941805
- type: nauc_precision_at_10_diff1
value: 29.57198931620274
- type: nauc_precision_at_10_max
value: 35.25411892642657
- type: nauc_precision_at_10_std
value: 10.888581830276264
- type: nauc_precision_at_1_diff1
value: 48.35468594232892
- type: nauc_precision_at_1_max
value: 36.961573378969376
- type: nauc_precision_at_1_std
value: 4.676168765221803
- type: nauc_precision_at_20_diff1
value: 26.899418278663152
- type: nauc_precision_at_20_max
value: 33.33984309930851
- type: nauc_precision_at_20_std
value: 10.265426034854858
- type: nauc_precision_at_3_diff1
value: 36.3516004410158
- type: nauc_precision_at_3_max
value: 36.38454388482908
- type: nauc_precision_at_3_std
value: 6.387721401060522
- type: nauc_precision_at_5_diff1
value: 33.78360486342633
- type: nauc_precision_at_5_max
value: 36.80099740935766
- type: nauc_precision_at_5_std
value: 9.608552479332767
- type: nauc_recall_at_1000_diff1
value: 22.86824125570728
- type: nauc_recall_at_1000_max
value: 26.919634548739946
- type: nauc_recall_at_1000_std
value: 22.890508862462127
- type: nauc_recall_at_100_diff1
value: 23.82352298540404
- type: nauc_recall_at_100_max
value: 29.284361970607552
- type: nauc_recall_at_100_std
value: 12.320881618564345
- type: nauc_recall_at_10_diff1
value: 33.13454578667508
- type: nauc_recall_at_10_max
value: 33.3715603672648
- type: nauc_recall_at_10_std
value: 8.041573457233307
- type: nauc_recall_at_1_diff1
value: 51.071763745176526
- type: nauc_recall_at_1_max
value: 36.354013439554656
- type: nauc_recall_at_1_std
value: 4.009623246187462
- type: nauc_recall_at_20_diff1
value: 31.914325899394303
- type: nauc_recall_at_20_max
value: 31.837899648967237
- type: nauc_recall_at_20_std
value: 7.142210811345996
- type: nauc_recall_at_3_diff1
value: 40.34198106489463
- type: nauc_recall_at_3_max
value: 35.339934791790505
- type: nauc_recall_at_3_std
value: 5.405846187725809
- type: nauc_recall_at_5_diff1
value: 37.80933481211865
- type: nauc_recall_at_5_max
value: 35.190301771222444
- type: nauc_recall_at_5_std
value: 7.413791949410964
- type: ndcg_at_1
value: 19.661
- type: ndcg_at_10
value: 26.869
- type: ndcg_at_100
value: 30.896
- type: ndcg_at_1000
value: 33.782000000000004
- type: ndcg_at_20
value: 28.218
- type: ndcg_at_3
value: 23.941000000000003
- type: ndcg_at_5
value: 25.341
- type: precision_at_1
value: 19.661
- type: precision_at_10
value: 4.011
- type: precision_at_100
value: 0.634
- type: precision_at_1000
value: 0.092
- type: precision_at_20
value: 2.328
- type: precision_at_3
value: 10.094
- type: precision_at_5
value: 6.893000000000001
- type: recall_at_1
value: 18.177
- type: recall_at_10
value: 34.942
- type: recall_at_100
value: 53.845
- type: recall_at_1000
value: 76.343
- type: recall_at_20
value: 40.022000000000006
- type: recall_at_3
value: 26.813
- type: recall_at_5
value: 30.312
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackMathematicaRetrieval (default)
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
split: test
type: mteb/cqadupstack-mathematica
metrics:
- type: main_score
value: 21.728
- type: map_at_1
value: 12.784
- type: map_at_10
value: 17.944
- type: map_at_100
value: 18.797
- type: map_at_1000
value: 18.926000000000002
- type: map_at_20
value: 18.387999999999998
- type: map_at_3
value: 16.265
- type: map_at_5
value: 16.983999999999998
- type: mrr_at_1
value: 16.044776119402986
- type: mrr_at_10
value: 21.803680012635233
- type: mrr_at_100
value: 22.575555542812165
- type: mrr_at_1000
value: 22.66901508901914
- type: mrr_at_20
value: 22.19854940073506
- type: mrr_at_3
value: 19.941956882255386
- type: mrr_at_5
value: 20.719320066334994
- type: nauc_map_at_1000_diff1
value: 37.95003279508669
- type: nauc_map_at_1000_max
value: 31.326475206055974
- type: nauc_map_at_1000_std
value: 4.664013633591596
- type: nauc_map_at_100_diff1
value: 37.96791591636629
- type: nauc_map_at_100_max
value: 31.28842831225577
- type: nauc_map_at_100_std
value: 4.589727393270644
- type: nauc_map_at_10_diff1
value: 38.507132083215595
- type: nauc_map_at_10_max
value: 31.169056023300783
- type: nauc_map_at_10_std
value: 4.368613661323174
- type: nauc_map_at_1_diff1
value: 44.35378845259204
- type: nauc_map_at_1_max
value: 31.252743839426383
- type: nauc_map_at_1_std
value: 3.168581843134213
- type: nauc_map_at_20_diff1
value: 38.187357484768896
- type: nauc_map_at_20_max
value: 31.2136407998411
- type: nauc_map_at_20_std
value: 4.333490585321438
- type: nauc_map_at_3_diff1
value: 39.330780048911656
- type: nauc_map_at_3_max
value: 31.37578299086863
- type: nauc_map_at_3_std
value: 4.188856474447449
- type: nauc_map_at_5_diff1
value: 39.20107804915463
- type: nauc_map_at_5_max
value: 31.392530920166408
- type: nauc_map_at_5_std
value: 4.056921377738636
- type: nauc_mrr_at_1000_diff1
value: 36.73182170905758
- type: nauc_mrr_at_1000_max
value: 31.05432855743618
- type: nauc_mrr_at_1000_std
value: 5.785548465518801
- type: nauc_mrr_at_100_diff1
value: 36.72314859513544
- type: nauc_mrr_at_100_max
value: 31.05422735924907
- type: nauc_mrr_at_100_std
value: 5.775106667651726
- type: nauc_mrr_at_10_diff1
value: 37.06378660745046
- type: nauc_mrr_at_10_max
value: 31.020085446718422
- type: nauc_mrr_at_10_std
value: 5.590867271501978
- type: nauc_mrr_at_1_diff1
value: 44.00103632989039
- type: nauc_mrr_at_1_max
value: 31.94201788773754
- type: nauc_mrr_at_1_std
value: 4.713263837383352
- type: nauc_mrr_at_20_diff1
value: 36.77294192719807
- type: nauc_mrr_at_20_max
value: 31.014175572052373
- type: nauc_mrr_at_20_std
value: 5.658270027362274
- type: nauc_mrr_at_3_diff1
value: 38.35434515359319
- type: nauc_mrr_at_3_max
value: 31.575718211424807
- type: nauc_mrr_at_3_std
value: 5.88624291840881
- type: nauc_mrr_at_5_diff1
value: 38.06890741693633
- type: nauc_mrr_at_5_max
value: 31.261036367423934
- type: nauc_mrr_at_5_std
value: 5.345199356889732
- type: nauc_ndcg_at_1000_diff1
value: 33.67924232501883
- type: nauc_ndcg_at_1000_max
value: 31.319148060296097
- type: nauc_ndcg_at_1000_std
value: 7.2514381178779095
- type: nauc_ndcg_at_100_diff1
value: 34.100629993327196
- type: nauc_ndcg_at_100_max
value: 31.278013773761526
- type: nauc_ndcg_at_100_std
value: 6.565787610461661
- type: nauc_ndcg_at_10_diff1
value: 35.84993832912045
- type: nauc_ndcg_at_10_max
value: 30.64709847569812
- type: nauc_ndcg_at_10_std
value: 5.313392670674671
- type: nauc_ndcg_at_1_diff1
value: 44.00103632989039
- type: nauc_ndcg_at_1_max
value: 31.94201788773754
- type: nauc_ndcg_at_1_std
value: 4.713263837383352
- type: nauc_ndcg_at_20_diff1
value: 34.78453662069707
- type: nauc_ndcg_at_20_max
value: 30.683192809444787
- type: nauc_ndcg_at_20_std
value: 5.269616192797054
- type: nauc_ndcg_at_3_diff1
value: 37.62191304337262
- type: nauc_ndcg_at_3_max
value: 31.623265872974997
- type: nauc_ndcg_at_3_std
value: 5.080503409806257
- type: nauc_ndcg_at_5_diff1
value: 37.56462137510564
- type: nauc_ndcg_at_5_max
value: 31.37187196099152
- type: nauc_ndcg_at_5_std
value: 4.624284925129432
- type: nauc_precision_at_1000_diff1
value: -3.146851839401258
- type: nauc_precision_at_1000_max
value: 10.977182155908867
- type: nauc_precision_at_1000_std
value: 10.160444053718093
- type: nauc_precision_at_100_diff1
value: 12.001129161175122
- type: nauc_precision_at_100_max
value: 25.308778188675856
- type: nauc_precision_at_100_std
value: 12.509597417847507
- type: nauc_precision_at_10_diff1
value: 25.664686247632844
- type: nauc_precision_at_10_max
value: 28.770491403202108
- type: nauc_precision_at_10_std
value: 6.536821071514153
- type: nauc_precision_at_1_diff1
value: 44.00103632989039
- type: nauc_precision_at_1_max
value: 31.94201788773754
- type: nauc_precision_at_1_std
value: 4.713263837383352
- type: nauc_precision_at_20_diff1
value: 21.77768613115255
- type: nauc_precision_at_20_max
value: 27.845144717145004
- type: nauc_precision_at_20_std
value: 6.320507052087349
- type: nauc_precision_at_3_diff1
value: 33.18575428477401
- type: nauc_precision_at_3_max
value: 31.92822455113195
- type: nauc_precision_at_3_std
value: 6.360274710244685
- type: nauc_precision_at_5_diff1
value: 32.148298285156365
- type: nauc_precision_at_5_max
value: 31.154123802087337
- type: nauc_precision_at_5_std
value: 5.0693172025299384
- type: nauc_recall_at_1000_diff1
value: 16.64989458294879
- type: nauc_recall_at_1000_max
value: 27.67387417287623
- type: nauc_recall_at_1000_std
value: 16.579859975118456
- type: nauc_recall_at_100_diff1
value: 23.24214920953384
- type: nauc_recall_at_100_max
value: 28.655454163454536
- type: nauc_recall_at_100_std
value: 10.354544546339836
- type: nauc_recall_at_10_diff1
value: 29.382576143547634
- type: nauc_recall_at_10_max
value: 27.625736210770256
- type: nauc_recall_at_10_std
value: 6.656383905649591
- type: nauc_recall_at_1_diff1
value: 44.35378845259204
- type: nauc_recall_at_1_max
value: 31.252743839426383
- type: nauc_recall_at_1_std
value: 3.168581843134213
- type: nauc_recall_at_20_diff1
value: 25.739220032350257
- type: nauc_recall_at_20_max
value: 27.108433769048872
- type: nauc_recall_at_20_std
value: 6.22385370891178
- type: nauc_recall_at_3_diff1
value: 33.80249079929206
- type: nauc_recall_at_3_max
value: 30.481055725978024
- type: nauc_recall_at_3_std
value: 5.346572769135906
- type: nauc_recall_at_5_diff1
value: 33.067070690653495
- type: nauc_recall_at_5_max
value: 29.433391080158426
- type: nauc_recall_at_5_std
value: 4.517583549915587
- type: ndcg_at_1
value: 16.045
- type: ndcg_at_10
value: 21.728
- type: ndcg_at_100
value: 26.106
- type: ndcg_at_1000
value: 29.439
- type: ndcg_at_20
value: 23.25
- type: ndcg_at_3
value: 18.297
- type: ndcg_at_5
value: 19.438
- type: precision_at_1
value: 16.045
- type: precision_at_10
value: 3.9800000000000004
- type: precision_at_100
value: 0.703
- type: precision_at_1000
value: 0.11100000000000002
- type: precision_at_20
value: 2.394
- type: precision_at_3
value: 8.581999999999999
- type: precision_at_5
value: 5.995
- type: recall_at_1
value: 12.784
- type: recall_at_10
value: 29.749
- type: recall_at_100
value: 49.412
- type: recall_at_1000
value: 73.778
- type: recall_at_20
value: 35.317
- type: recall_at_3
value: 20.05
- type: recall_at_5
value: 22.996
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackPhysicsRetrieval (default)
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
split: test
type: mteb/cqadupstack-physics
metrics:
- type: main_score
value: 31.362000000000002
- type: map_at_1
value: 21.522
- type: map_at_10
value: 27.397
- type: map_at_100
value: 28.374
- type: map_at_1000
value: 28.496
- type: map_at_20
value: 27.915
- type: map_at_3
value: 25.330000000000002
- type: map_at_5
value: 26.622
- type: mrr_at_1
value: 26.564003849855634
- type: mrr_at_10
value: 32.16665138946176
- type: mrr_at_100
value: 32.97230640563016
- type: mrr_at_1000
value: 33.050704711262505
- type: mrr_at_20
value: 32.63506041957575
- type: mrr_at_3
value: 30.25344882900225
- type: mrr_at_5
value: 31.470965672120617
- type: nauc_map_at_1000_diff1
value: 52.59688767459125
- type: nauc_map_at_1000_max
value: 49.49548390354128
- type: nauc_map_at_1000_std
value: 10.115628251616295
- type: nauc_map_at_100_diff1
value: 52.61576203524935
- type: nauc_map_at_100_max
value: 49.48503434991592
- type: nauc_map_at_100_std
value: 10.111348427919129
- type: nauc_map_at_10_diff1
value: 53.07085205321233
- type: nauc_map_at_10_max
value: 49.40570059340953
- type: nauc_map_at_10_std
value: 9.54214632320025
- type: nauc_map_at_1_diff1
value: 60.33424890291046
- type: nauc_map_at_1_max
value: 48.81194468082383
- type: nauc_map_at_1_std
value: 5.36430977601789
- type: nauc_map_at_20_diff1
value: 52.773568860496354
- type: nauc_map_at_20_max
value: 49.426884901362584
- type: nauc_map_at_20_std
value: 9.896169869423503
- type: nauc_map_at_3_diff1
value: 54.99635374447428
- type: nauc_map_at_3_max
value: 49.25542338008625
- type: nauc_map_at_3_std
value: 7.920730174530154
- type: nauc_map_at_5_diff1
value: 53.68219394103163
- type: nauc_map_at_5_max
value: 49.17446366918848
- type: nauc_map_at_5_std
value: 8.821731461186905
- type: nauc_mrr_at_1000_diff1
value: 51.099384685054616
- type: nauc_mrr_at_1000_max
value: 48.54414875311902
- type: nauc_mrr_at_1000_std
value: 9.780074869585286
- type: nauc_mrr_at_100_diff1
value: 51.08502477800608
- type: nauc_mrr_at_100_max
value: 48.542070464175055
- type: nauc_mrr_at_100_std
value: 9.783011896915687
- type: nauc_mrr_at_10_diff1
value: 51.382177619679204
- type: nauc_mrr_at_10_max
value: 48.54890111812492
- type: nauc_mrr_at_10_std
value: 9.494226343992503
- type: nauc_mrr_at_1_diff1
value: 56.85885674241863
- type: nauc_mrr_at_1_max
value: 48.534841672238585
- type: nauc_mrr_at_1_std
value: 7.045749638810437
- type: nauc_mrr_at_20_diff1
value: 51.146039318789846
- type: nauc_mrr_at_20_max
value: 48.53675895185779
- type: nauc_mrr_at_20_std
value: 9.696565280887011
- type: nauc_mrr_at_3_diff1
value: 52.886914910351734
- type: nauc_mrr_at_3_max
value: 48.73402755122022
- type: nauc_mrr_at_3_std
value: 8.463221573475908
- type: nauc_mrr_at_5_diff1
value: 51.81731850552879
- type: nauc_mrr_at_5_max
value: 48.579649148508494
- type: nauc_mrr_at_5_std
value: 9.41815937454225
- type: nauc_ndcg_at_1000_diff1
value: 48.320439093318356
- type: nauc_ndcg_at_1000_max
value: 49.43059637755541
- type: nauc_ndcg_at_1000_std
value: 13.17421117129521
- type: nauc_ndcg_at_100_diff1
value: 48.269268235554044
- type: nauc_ndcg_at_100_max
value: 49.34734693138846
- type: nauc_ndcg_at_100_std
value: 13.33552837762993
- type: nauc_ndcg_at_10_diff1
value: 49.98164427597868
- type: nauc_ndcg_at_10_max
value: 49.34516846968058
- type: nauc_ndcg_at_10_std
value: 11.25989970999377
- type: nauc_ndcg_at_1_diff1
value: 56.85885674241863
- type: nauc_ndcg_at_1_max
value: 48.534841672238585
- type: nauc_ndcg_at_1_std
value: 7.045749638810437
- type: nauc_ndcg_at_20_diff1
value: 49.090030521758024
- type: nauc_ndcg_at_20_max
value: 49.31796649039315
- type: nauc_ndcg_at_20_std
value: 12.237851926677221
- type: nauc_ndcg_at_3_diff1
value: 52.63555826739571
- type: nauc_ndcg_at_3_max
value: 49.19040862536399
- type: nauc_ndcg_at_3_std
value: 8.743373927918615
- type: nauc_ndcg_at_5_diff1
value: 50.90837252531234
- type: nauc_ndcg_at_5_max
value: 49.012102442680046
- type: nauc_ndcg_at_5_std
value: 10.23622790560761
- type: nauc_precision_at_1000_diff1
value: -7.907291444067378
- type: nauc_precision_at_1000_max
value: 10.551673138545729
- type: nauc_precision_at_1000_std
value: 9.346444460189003
- type: nauc_precision_at_100_diff1
value: 7.916501406079248
- type: nauc_precision_at_100_max
value: 27.483613003895197
- type: nauc_precision_at_100_std
value: 18.76421395657329
- type: nauc_precision_at_10_diff1
value: 26.79009275910224
- type: nauc_precision_at_10_max
value: 41.84469020826695
- type: nauc_precision_at_10_std
value: 17.48055617040187
- type: nauc_precision_at_1_diff1
value: 56.85885674241863
- type: nauc_precision_at_1_max
value: 48.534841672238585
- type: nauc_precision_at_1_std
value: 7.045749638810437
- type: nauc_precision_at_20_diff1
value: 20.94274635916315
- type: nauc_precision_at_20_max
value: 38.68527168262968
- type: nauc_precision_at_20_std
value: 19.52732371032245
- type: nauc_precision_at_3_diff1
value: 39.632955443069804
- type: nauc_precision_at_3_max
value: 45.32402555048772
- type: nauc_precision_at_3_std
value: 11.326283322156945
- type: nauc_precision_at_5_diff1
value: 32.91121403324715
- type: nauc_precision_at_5_max
value: 43.74183654588345
- type: nauc_precision_at_5_std
value: 14.704405107052429
- type: nauc_recall_at_1000_diff1
value: 24.449918058702696
- type: nauc_recall_at_1000_max
value: 42.19385988290269
- type: nauc_recall_at_1000_std
value: 27.970662815885987
- type: nauc_recall_at_100_diff1
value: 31.75049770052644
- type: nauc_recall_at_100_max
value: 44.42578620612963
- type: nauc_recall_at_100_std
value: 23.842848082939103
- type: nauc_recall_at_10_diff1
value: 41.105148509808075
- type: nauc_recall_at_10_max
value: 47.17211207978789
- type: nauc_recall_at_10_std
value: 15.279920198725518
- type: nauc_recall_at_1_diff1
value: 60.33424890291046
- type: nauc_recall_at_1_max
value: 48.81194468082383
- type: nauc_recall_at_1_std
value: 5.36430977601789
- type: nauc_recall_at_20_diff1
value: 37.228002397984305
- type: nauc_recall_at_20_max
value: 45.77317403012792
- type: nauc_recall_at_20_std
value: 17.986553641491952
- type: nauc_recall_at_3_diff1
value: 49.828715929251175
- type: nauc_recall_at_3_max
value: 49.01281750728966
- type: nauc_recall_at_3_std
value: 10.319882588842756
- type: nauc_recall_at_5_diff1
value: 44.56189814238881
- type: nauc_recall_at_5_max
value: 47.19512580179057
- type: nauc_recall_at_5_std
value: 12.807907369220995
- type: ndcg_at_1
value: 26.564
- type: ndcg_at_10
value: 31.362000000000002
- type: ndcg_at_100
value: 36.098
- type: ndcg_at_1000
value: 38.927
- type: ndcg_at_20
value: 33.086
- type: ndcg_at_3
value: 28.000999999999998
- type: ndcg_at_5
value: 29.871
- type: precision_at_1
value: 26.564
- type: precision_at_10
value: 5.38
- type: precision_at_100
value: 0.9039999999999999
- type: precision_at_1000
value: 0.129
- type: precision_at_20
value: 3.215
- type: precision_at_3
value: 12.928999999999998
- type: precision_at_5
value: 9.182
- type: recall_at_1
value: 21.522
- type: recall_at_10
value: 38.778
- type: recall_at_100
value: 59.57
- type: recall_at_1000
value: 79.318
- type: recall_at_20
value: 44.974
- type: recall_at_3
value: 29.142000000000003
- type: recall_at_5
value: 34.142
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackProgrammersRetrieval (default)
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
split: test
type: mteb/cqadupstack-programmers
metrics:
- type: main_score
value: 30.611
- type: map_at_1
value: 20.044
- type: map_at_10
value: 26.819
- type: map_at_100
value: 27.79
- type: map_at_1000
value: 27.889999999999997
- type: map_at_20
value: 27.365000000000002
- type: map_at_3
value: 25.012
- type: map_at_5
value: 26.045
- type: mrr_at_1
value: 24.771689497716896
- type: mrr_at_10
value: 30.936571356091886
- type: mrr_at_100
value: 31.812872551313287
- type: mrr_at_1000
value: 31.880398536845707
- type: mrr_at_20
value: 31.445758027356035
- type: mrr_at_3
value: 29.33789954337901
- type: mrr_at_5
value: 30.291095890410947
- type: nauc_map_at_1000_diff1
value: 56.77795203729021
- type: nauc_map_at_1000_max
value: 50.228903814053425
- type: nauc_map_at_1000_std
value: 8.282984319702154
- type: nauc_map_at_100_diff1
value: 56.750094825915
- type: nauc_map_at_100_max
value: 50.20535641719778
- type: nauc_map_at_100_std
value: 8.259689362917367
- type: nauc_map_at_10_diff1
value: 57.36310524366185
- type: nauc_map_at_10_max
value: 50.418894761062326
- type: nauc_map_at_10_std
value: 7.6104755737897785
- type: nauc_map_at_1_diff1
value: 64.66157606359778
- type: nauc_map_at_1_max
value: 48.711361165288125
- type: nauc_map_at_1_std
value: 2.610460584994949
- type: nauc_map_at_20_diff1
value: 56.96394361697854
- type: nauc_map_at_20_max
value: 50.22074366032774
- type: nauc_map_at_20_std
value: 7.987582744402786
- type: nauc_map_at_3_diff1
value: 58.30755542137215
- type: nauc_map_at_3_max
value: 49.77444912203085
- type: nauc_map_at_3_std
value: 6.112464344603168
- type: nauc_map_at_5_diff1
value: 57.197612503845875
- type: nauc_map_at_5_max
value: 49.86870173806947
- type: nauc_map_at_5_std
value: 6.741251739454883
- type: nauc_mrr_at_1000_diff1
value: 56.55345796340776
- type: nauc_mrr_at_1000_max
value: 51.984711889193214
- type: nauc_mrr_at_1000_std
value: 11.330935517234703
- type: nauc_mrr_at_100_diff1
value: 56.53520001259168
- type: nauc_mrr_at_100_max
value: 51.971724449016065
- type: nauc_mrr_at_100_std
value: 11.326566973175943
- type: nauc_mrr_at_10_diff1
value: 57.02811409284357
- type: nauc_mrr_at_10_max
value: 52.17421961618363
- type: nauc_mrr_at_10_std
value: 10.922170784918812
- type: nauc_mrr_at_1_diff1
value: 63.398417335709034
- type: nauc_mrr_at_1_max
value: 52.76393807206049
- type: nauc_mrr_at_1_std
value: 7.987883095781358
- type: nauc_mrr_at_20_diff1
value: 56.66914685804212
- type: nauc_mrr_at_20_max
value: 52.034098755328216
- type: nauc_mrr_at_20_std
value: 11.201609751277623
- type: nauc_mrr_at_3_diff1
value: 57.76424743500629
- type: nauc_mrr_at_3_max
value: 52.21692819835331
- type: nauc_mrr_at_3_std
value: 10.351547071966488
- type: nauc_mrr_at_5_diff1
value: 56.917790279249225
- type: nauc_mrr_at_5_max
value: 51.78687350716348
- type: nauc_mrr_at_5_std
value: 10.356241647673261
- type: nauc_ndcg_at_1000_diff1
value: 52.91063006408778
- type: nauc_ndcg_at_1000_max
value: 50.30527611594974
- type: nauc_ndcg_at_1000_std
value: 12.61486210785366
- type: nauc_ndcg_at_100_diff1
value: 52.51126278764645
- type: nauc_ndcg_at_100_max
value: 49.9198621161366
- type: nauc_ndcg_at_100_std
value: 12.48612827348202
- type: nauc_ndcg_at_10_diff1
value: 55.31262109361159
- type: nauc_ndcg_at_10_max
value: 50.95121518060335
- type: nauc_ndcg_at_10_std
value: 9.99289135242688
- type: nauc_ndcg_at_1_diff1
value: 63.398417335709034
- type: nauc_ndcg_at_1_max
value: 52.76393807206049
- type: nauc_ndcg_at_1_std
value: 7.987883095781358
- type: nauc_ndcg_at_20_diff1
value: 53.92230329753415
- type: nauc_ndcg_at_20_max
value: 50.272129616480456
- type: nauc_ndcg_at_20_std
value: 11.048172123886694
- type: nauc_ndcg_at_3_diff1
value: 56.824616064178215
- type: nauc_ndcg_at_3_max
value: 50.379870694016596
- type: nauc_ndcg_at_3_std
value: 7.948147030055066
- type: nauc_ndcg_at_5_diff1
value: 55.080128128233774
- type: nauc_ndcg_at_5_max
value: 49.982141089686074
- type: nauc_ndcg_at_5_std
value: 8.260960076197717
- type: nauc_precision_at_1000_diff1
value: 1.8819061790883571
- type: nauc_precision_at_1000_max
value: 14.62991850470928
- type: nauc_precision_at_1000_std
value: 15.308588820796068
- type: nauc_precision_at_100_diff1
value: 15.71517907419952
- type: nauc_precision_at_100_max
value: 30.9161164726617
- type: nauc_precision_at_100_std
value: 23.928744380434246
- type: nauc_precision_at_10_diff1
value: 40.086577642823556
- type: nauc_precision_at_10_max
value: 48.9731405153383
- type: nauc_precision_at_10_std
value: 20.225991151869817
- type: nauc_precision_at_1_diff1
value: 63.398417335709034
- type: nauc_precision_at_1_max
value: 52.76393807206049
- type: nauc_precision_at_1_std
value: 7.987883095781358
- type: nauc_precision_at_20_diff1
value: 32.99649371260927
- type: nauc_precision_at_20_max
value: 43.9358941187044
- type: nauc_precision_at_20_std
value: 22.96013196641051
- type: nauc_precision_at_3_diff1
value: 47.96996864044809
- type: nauc_precision_at_3_max
value: 50.55165745431339
- type: nauc_precision_at_3_std
value: 14.336246257412755
- type: nauc_precision_at_5_diff1
value: 42.43305463117832
- type: nauc_precision_at_5_max
value: 49.04724446709852
- type: nauc_precision_at_5_std
value: 15.650628037765497
- type: nauc_recall_at_1000_diff1
value: 27.81489607198872
- type: nauc_recall_at_1000_max
value: 42.635078937347124
- type: nauc_recall_at_1000_std
value: 28.28899115429
- type: nauc_recall_at_100_diff1
value: 33.31240744474597
- type: nauc_recall_at_100_max
value: 41.92541065720442
- type: nauc_recall_at_100_std
value: 22.810710155447726
- type: nauc_recall_at_10_diff1
value: 48.78704799247861
- type: nauc_recall_at_10_max
value: 49.17510660960629
- type: nauc_recall_at_10_std
value: 13.003749357577519
- type: nauc_recall_at_1_diff1
value: 64.66157606359778
- type: nauc_recall_at_1_max
value: 48.711361165288125
- type: nauc_recall_at_1_std
value: 2.610460584994949
- type: nauc_recall_at_20_diff1
value: 43.12305614808684
- type: nauc_recall_at_20_max
value: 45.86509269523116
- type: nauc_recall_at_20_std
value: 15.945979676808147
- type: nauc_recall_at_3_diff1
value: 52.266013736150974
- type: nauc_recall_at_3_max
value: 48.20329627231183
- type: nauc_recall_at_3_std
value: 7.662445739786749
- type: nauc_recall_at_5_diff1
value: 48.007088225221246
- type: nauc_recall_at_5_max
value: 46.87942175385594
- type: nauc_recall_at_5_std
value: 8.76803938978432
- type: ndcg_at_1
value: 24.772
- type: ndcg_at_10
value: 30.611
- type: ndcg_at_100
value: 35.315999999999995
- type: ndcg_at_1000
value: 37.846000000000004
- type: ndcg_at_20
value: 32.429
- type: ndcg_at_3
value: 27.63
- type: ndcg_at_5
value: 29.066
- type: precision_at_1
value: 24.772
- type: precision_at_10
value: 5.183
- type: precision_at_100
value: 0.873
- type: precision_at_1000
value: 0.125
- type: precision_at_20
value: 3.088
- type: precision_at_3
value: 12.937999999999999
- type: precision_at_5
value: 8.927
- type: recall_at_1
value: 20.044
- type: recall_at_10
value: 38.073
- type: recall_at_100
value: 58.632
- type: recall_at_1000
value: 76.509
- type: recall_at_20
value: 44.651
- type: recall_at_3
value: 29.660999999999998
- type: recall_at_5
value: 33.555
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackRetrieval (default)
revision: CQADupstackRetrieval_is_a_combined_dataset
split: test
type: CQADupstackRetrieval_is_a_combined_dataset
metrics:
- type: main_score
value: 29.904000000000003
- type: ndcg_at_10
value: 29.904000000000003
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackStatsRetrieval (default)
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
split: test
type: mteb/cqadupstack-stats
metrics:
- type: main_score
value: 27.844
- type: map_at_1
value: 20.513
- type: map_at_10
value: 24.859
- type: map_at_100
value: 25.618000000000002
- type: map_at_1000
value: 25.694
- type: map_at_20
value: 25.195
- type: map_at_3
value: 23.634
- type: map_at_5
value: 24.083
- type: mrr_at_1
value: 22.85276073619632
- type: mrr_at_10
value: 27.365493232057652
- type: mrr_at_100
value: 28.059687087154796
- type: mrr_at_1000
value: 28.126281152513645
- type: mrr_at_20
value: 27.669375141110137
- type: mrr_at_3
value: 26.20143149284254
- type: mrr_at_5
value: 26.577198364008186
- type: nauc_map_at_1000_diff1
value: 58.0923988657438
- type: nauc_map_at_1000_max
value: 39.49776904657763
- type: nauc_map_at_1000_std
value: 1.438859276174183
- type: nauc_map_at_100_diff1
value: 58.10154425542812
- type: nauc_map_at_100_max
value: 39.46324052006619
- type: nauc_map_at_100_std
value: 1.3890820890372753
- type: nauc_map_at_10_diff1
value: 58.5332816613564
- type: nauc_map_at_10_max
value: 39.45154561291042
- type: nauc_map_at_10_std
value: 0.9027921971290199
- type: nauc_map_at_1_diff1
value: 62.56595068686164
- type: nauc_map_at_1_max
value: 41.83674415695135
- type: nauc_map_at_1_std
value: 0.0645259857866332
- type: nauc_map_at_20_diff1
value: 58.42227923867406
- type: nauc_map_at_20_max
value: 39.4909671277415
- type: nauc_map_at_20_std
value: 1.1087675448202337
- type: nauc_map_at_3_diff1
value: 59.80877486724449
- type: nauc_map_at_3_max
value: 39.44386269848118
- type: nauc_map_at_3_std
value: -0.48978195872711355
- type: nauc_map_at_5_diff1
value: 59.74549487724378
- type: nauc_map_at_5_max
value: 39.46911071607922
- type: nauc_map_at_5_std
value: -0.2054925595619289
- type: nauc_mrr_at_1000_diff1
value: 57.22992893520331
- type: nauc_mrr_at_1000_max
value: 39.245802225236126
- type: nauc_mrr_at_1000_std
value: 1.4419858232822822
- type: nauc_mrr_at_100_diff1
value: 57.240863535228094
- type: nauc_mrr_at_100_max
value: 39.23508417147578
- type: nauc_mrr_at_100_std
value: 1.4179374600097412
- type: nauc_mrr_at_10_diff1
value: 57.60090554706999
- type: nauc_mrr_at_10_max
value: 39.20522608560329
- type: nauc_mrr_at_10_std
value: 1.087952585482026
- type: nauc_mrr_at_1_diff1
value: 61.959334311281744
- type: nauc_mrr_at_1_max
value: 42.02253807367881
- type: nauc_mrr_at_1_std
value: 0.3652657393695138
- type: nauc_mrr_at_20_diff1
value: 57.42893340359555
- type: nauc_mrr_at_20_max
value: 39.24222177257406
- type: nauc_mrr_at_20_std
value: 1.1924382031739464
- type: nauc_mrr_at_3_diff1
value: 58.62920326596276
- type: nauc_mrr_at_3_max
value: 39.11872316960711
- type: nauc_mrr_at_3_std
value: -0.19139543362787656
- type: nauc_mrr_at_5_diff1
value: 58.72776007304141
- type: nauc_mrr_at_5_max
value: 39.19114294986413
- type: nauc_mrr_at_5_std
value: -0.04284085490819686
- type: nauc_ndcg_at_1000_diff1
value: 53.53402253546664
- type: nauc_ndcg_at_1000_max
value: 39.864833616084574
- type: nauc_ndcg_at_1000_std
value: 6.0573552449257235
- type: nauc_ndcg_at_100_diff1
value: 53.70663536580926
- type: nauc_ndcg_at_100_max
value: 38.89279345762969
- type: nauc_ndcg_at_100_std
value: 5.078041324095517
- type: nauc_ndcg_at_10_diff1
value: 55.88558776920169
- type: nauc_ndcg_at_10_max
value: 38.576588113891695
- type: nauc_ndcg_at_10_std
value: 2.353727524868291
- type: nauc_ndcg_at_1_diff1
value: 61.959334311281744
- type: nauc_ndcg_at_1_max
value: 42.02253807367881
- type: nauc_ndcg_at_1_std
value: 0.3652657393695138
- type: nauc_ndcg_at_20_diff1
value: 55.43183888815839
- type: nauc_ndcg_at_20_max
value: 38.77692588511735
- type: nauc_ndcg_at_20_std
value: 3.0554783814959703
- type: nauc_ndcg_at_3_diff1
value: 58.11064498159326
- type: nauc_ndcg_at_3_max
value: 38.10766803905449
- type: nauc_ndcg_at_3_std
value: -0.7498834791332512
- type: nauc_ndcg_at_5_diff1
value: 58.40382932520117
- type: nauc_ndcg_at_5_max
value: 38.44740384423254
- type: nauc_ndcg_at_5_std
value: -0.2751100258658748
- type: nauc_precision_at_1000_diff1
value: 6.170519960423255
- type: nauc_precision_at_1000_max
value: 25.861700118167274
- type: nauc_precision_at_1000_std
value: 21.252375305577324
- type: nauc_precision_at_100_diff1
value: 23.83932564925894
- type: nauc_precision_at_100_max
value: 30.9046567399569
- type: nauc_precision_at_100_std
value: 17.771919583124987
- type: nauc_precision_at_10_diff1
value: 42.59052382404775
- type: nauc_precision_at_10_max
value: 33.03591527422188
- type: nauc_precision_at_10_std
value: 6.034666442072681
- type: nauc_precision_at_1_diff1
value: 61.959334311281744
- type: nauc_precision_at_1_max
value: 42.02253807367881
- type: nauc_precision_at_1_std
value: 0.3652657393695138
- type: nauc_precision_at_20_diff1
value: 38.82622844929183
- type: nauc_precision_at_20_max
value: 33.101913493165085
- type: nauc_precision_at_20_std
value: 8.790703023502918
- type: nauc_precision_at_3_diff1
value: 52.53750604424028
- type: nauc_precision_at_3_max
value: 34.62182030721166
- type: nauc_precision_at_3_std
value: -0.4752345033778231
- type: nauc_precision_at_5_diff1
value: 50.588894261265395
- type: nauc_precision_at_5_max
value: 33.9141126886655
- type: nauc_precision_at_5_std
value: 0.4329659248962148
- type: nauc_recall_at_1000_diff1
value: 29.329091515084176
- type: nauc_recall_at_1000_max
value: 43.91061237189976
- type: nauc_recall_at_1000_std
value: 33.27518294715009
- type: nauc_recall_at_100_diff1
value: 36.755585803005886
- type: nauc_recall_at_100_max
value: 35.85937772290077
- type: nauc_recall_at_100_std
value: 19.437598135621656
- type: nauc_recall_at_10_diff1
value: 47.88480637900114
- type: nauc_recall_at_10_max
value: 35.617046526273135
- type: nauc_recall_at_10_std
value: 6.704114982627122
- type: nauc_recall_at_1_diff1
value: 62.56595068686164
- type: nauc_recall_at_1_max
value: 41.83674415695135
- type: nauc_recall_at_1_std
value: 0.0645259857866332
- type: nauc_recall_at_20_diff1
value: 46.19852363710199
- type: nauc_recall_at_20_max
value: 36.00423209498212
- type: nauc_recall_at_20_std
value: 8.8597610261735
- type: nauc_recall_at_3_diff1
value: 55.76604837913488
- type: nauc_recall_at_3_max
value: 35.62281066497774
- type: nauc_recall_at_3_std
value: -1.380737007802526
- type: nauc_recall_at_5_diff1
value: 55.591415778141716
- type: nauc_recall_at_5_max
value: 35.864379736339394
- type: nauc_recall_at_5_std
value: -0.2731705875996521
- type: ndcg_at_1
value: 22.853
- type: ndcg_at_10
value: 27.844
- type: ndcg_at_100
value: 32.054
- type: ndcg_at_1000
value: 34.387
- type: ndcg_at_20
value: 28.996
- type: ndcg_at_3
value: 25.447999999999997
- type: ndcg_at_5
value: 26.084000000000003
- type: precision_at_1
value: 22.853
- type: precision_at_10
value: 4.109999999999999
- type: precision_at_100
value: 0.6689999999999999
- type: precision_at_1000
value: 0.094
- type: precision_at_20
value: 2.339
- type: precision_at_3
value: 10.634
- type: precision_at_5
value: 6.902
- type: recall_at_1
value: 20.513
- type: recall_at_10
value: 34.714
- type: recall_at_100
value: 55.044000000000004
- type: recall_at_1000
value: 72.79299999999999
- type: recall_at_20
value: 39.098
- type: recall_at_3
value: 27.589999999999996
- type: recall_at_5
value: 29.332
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackTexRetrieval (default)
revision: 46989137a86843e03a6195de44b09deda022eec7
split: test
type: mteb/cqadupstack-tex
metrics:
- type: main_score
value: 22.158
- type: map_at_1
value: 14.234
- type: map_at_10
value: 19.029
- type: map_at_100
value: 19.698999999999998
- type: map_at_1000
value: 19.799
- type: map_at_20
value: 19.352
- type: map_at_3
value: 17.569000000000003
- type: map_at_5
value: 18.376
- type: mrr_at_1
value: 16.689607708189953
- type: mrr_at_10
value: 21.773342509749956
- type: mrr_at_100
value: 22.47549878924445
- type: mrr_at_1000
value: 22.550526256216752
- type: mrr_at_20
value: 22.13545007928467
- type: mrr_at_3
value: 20.291351227345725
- type: mrr_at_5
value: 21.094861206698788
- type: nauc_map_at_1000_diff1
value: 40.179576852413554
- type: nauc_map_at_1000_max
value: 37.26120147930247
- type: nauc_map_at_1000_std
value: 8.13006546084448
- type: nauc_map_at_100_diff1
value: 40.17714216412655
- type: nauc_map_at_100_max
value: 37.24762242407401
- type: nauc_map_at_100_std
value: 8.083691024610811
- type: nauc_map_at_10_diff1
value: 40.43131771246876
- type: nauc_map_at_10_max
value: 37.16329719233527
- type: nauc_map_at_10_std
value: 7.7252792661257255
- type: nauc_map_at_1_diff1
value: 48.29794914435358
- type: nauc_map_at_1_max
value: 37.62257118560468
- type: nauc_map_at_1_std
value: 4.786506895640649
- type: nauc_map_at_20_diff1
value: 40.325746610205755
- type: nauc_map_at_20_max
value: 37.1978051980932
- type: nauc_map_at_20_std
value: 7.858608782830421
- type: nauc_map_at_3_diff1
value: 42.08959990310158
- type: nauc_map_at_3_max
value: 37.341138179232296
- type: nauc_map_at_3_std
value: 6.728366839802072
- type: nauc_map_at_5_diff1
value: 41.06715366743666
- type: nauc_map_at_5_max
value: 37.246478933656796
- type: nauc_map_at_5_std
value: 7.316858433853442
- type: nauc_mrr_at_1000_diff1
value: 40.24906186828823
- type: nauc_mrr_at_1000_max
value: 37.72507978803392
- type: nauc_mrr_at_1000_std
value: 8.029307294065571
- type: nauc_mrr_at_100_diff1
value: 40.23526853440579
- type: nauc_mrr_at_100_max
value: 37.72020763714388
- type: nauc_mrr_at_100_std
value: 8.003595658733154
- type: nauc_mrr_at_10_diff1
value: 40.47982272945393
- type: nauc_mrr_at_10_max
value: 37.74949163847368
- type: nauc_mrr_at_10_std
value: 7.790504991653674
- type: nauc_mrr_at_1_diff1
value: 47.78839933089798
- type: nauc_mrr_at_1_max
value: 38.6059533361402
- type: nauc_mrr_at_1_std
value: 4.768871875484091
- type: nauc_mrr_at_20_diff1
value: 40.35529466284841
- type: nauc_mrr_at_20_max
value: 37.66043308235602
- type: nauc_mrr_at_20_std
value: 7.821535820635585
- type: nauc_mrr_at_3_diff1
value: 41.96110960386329
- type: nauc_mrr_at_3_max
value: 38.05282503377668
- type: nauc_mrr_at_3_std
value: 6.7837971383357605
- type: nauc_mrr_at_5_diff1
value: 40.99000202926007
- type: nauc_mrr_at_5_max
value: 37.90426104078837
- type: nauc_mrr_at_5_std
value: 7.383070824392375
- type: nauc_ndcg_at_1000_diff1
value: 36.35080188503686
- type: nauc_ndcg_at_1000_max
value: 37.533354704485916
- type: nauc_ndcg_at_1000_std
value: 12.240851975901647
- type: nauc_ndcg_at_100_diff1
value: 36.2865497577743
- type: nauc_ndcg_at_100_max
value: 37.127352376123426
- type: nauc_ndcg_at_100_std
value: 11.128543722024183
- type: nauc_ndcg_at_10_diff1
value: 37.754870398423414
- type: nauc_ndcg_at_10_max
value: 37.1826303982925
- type: nauc_ndcg_at_10_std
value: 9.474943711920515
- type: nauc_ndcg_at_1_diff1
value: 47.78839933089798
- type: nauc_ndcg_at_1_max
value: 38.6059533361402
- type: nauc_ndcg_at_1_std
value: 4.768871875484091
- type: nauc_ndcg_at_20_diff1
value: 37.38704153558385
- type: nauc_ndcg_at_20_max
value: 37.12769745209728
- type: nauc_ndcg_at_20_std
value: 9.788256573134186
- type: nauc_ndcg_at_3_diff1
value: 40.59763770525907
- type: nauc_ndcg_at_3_max
value: 37.79487738785331
- type: nauc_ndcg_at_3_std
value: 7.4988577781274435
- type: nauc_ndcg_at_5_diff1
value: 38.9663909407547
- type: nauc_ndcg_at_5_max
value: 37.44115701484436
- type: nauc_ndcg_at_5_std
value: 8.551584073883859
- type: nauc_precision_at_1000_diff1
value: 9.616540911306751
- type: nauc_precision_at_1000_max
value: 19.998238282494626
- type: nauc_precision_at_1000_std
value: 14.092277186107074
- type: nauc_precision_at_100_diff1
value: 19.885372189374742
- type: nauc_precision_at_100_max
value: 31.45358880325989
- type: nauc_precision_at_100_std
value: 16.463510466036496
- type: nauc_precision_at_10_diff1
value: 28.95841301518647
- type: nauc_precision_at_10_max
value: 36.781803245276194
- type: nauc_precision_at_10_std
value: 13.432773215377058
- type: nauc_precision_at_1_diff1
value: 47.78839933089798
- type: nauc_precision_at_1_max
value: 38.6059533361402
- type: nauc_precision_at_1_std
value: 4.768871875484091
- type: nauc_precision_at_20_diff1
value: 26.773518337425074
- type: nauc_precision_at_20_max
value: 35.92903140910233
- type: nauc_precision_at_20_std
value: 14.057129292615059
- type: nauc_precision_at_3_diff1
value: 35.40549562723217
- type: nauc_precision_at_3_max
value: 38.28419758227164
- type: nauc_precision_at_3_std
value: 9.048280179973776
- type: nauc_precision_at_5_diff1
value: 31.72752379888693
- type: nauc_precision_at_5_max
value: 37.40522255391838
- type: nauc_precision_at_5_std
value: 11.094065410830797
- type: nauc_recall_at_1000_diff1
value: 20.02685193454104
- type: nauc_recall_at_1000_max
value: 35.49046884558352
- type: nauc_recall_at_1000_std
value: 29.925161560313736
- type: nauc_recall_at_100_diff1
value: 23.18451498364251
- type: nauc_recall_at_100_max
value: 33.457782922773355
- type: nauc_recall_at_100_std
value: 19.36412617178296
- type: nauc_recall_at_10_diff1
value: 29.37078543338278
- type: nauc_recall_at_10_max
value: 34.72485078612908
- type: nauc_recall_at_10_std
value: 13.115270902006321
- type: nauc_recall_at_1_diff1
value: 48.29794914435358
- type: nauc_recall_at_1_max
value: 37.62257118560468
- type: nauc_recall_at_1_std
value: 4.786506895640649
- type: nauc_recall_at_20_diff1
value: 28.297194231787586
- type: nauc_recall_at_20_max
value: 34.36371177902483
- type: nauc_recall_at_20_std
value: 13.929281319745733
- type: nauc_recall_at_3_diff1
value: 35.987371367585894
- type: nauc_recall_at_3_max
value: 36.14164814954217
- type: nauc_recall_at_3_std
value: 8.94348815645971
- type: nauc_recall_at_5_diff1
value: 32.43919735989059
- type: nauc_recall_at_5_max
value: 35.40951237440019
- type: nauc_recall_at_5_std
value: 10.949869299555251
- type: ndcg_at_1
value: 16.689999999999998
- type: ndcg_at_10
value: 22.158
- type: ndcg_at_100
value: 25.790999999999997
- type: ndcg_at_1000
value: 28.65
- type: ndcg_at_20
value: 23.313
- type: ndcg_at_3
value: 19.414
- type: ndcg_at_5
value: 20.656
- type: precision_at_1
value: 16.689999999999998
- type: precision_at_10
value: 3.7470000000000003
- type: precision_at_100
value: 0.633
- type: precision_at_1000
value: 0.101
- type: precision_at_20
value: 2.2009999999999996
- type: precision_at_3
value: 8.844000000000001
- type: precision_at_5
value: 6.208
- type: recall_at_1
value: 14.234
- type: recall_at_10
value: 28.945999999999998
- type: recall_at_100
value: 45.812999999999995
- type: recall_at_1000
value: 67.007
- type: recall_at_20
value: 33.268
- type: recall_at_3
value: 21.258
- type: recall_at_5
value: 24.433
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackUnixRetrieval (default)
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
split: test
type: mteb/cqadupstack-unix
metrics:
- type: main_score
value: 28.954
- type: map_at_1
value: 20.124
- type: map_at_10
value: 25.739
- type: map_at_100
value: 26.512999999999998
- type: map_at_1000
value: 26.619999999999997
- type: map_at_20
value: 26.104
- type: map_at_3
value: 24.349
- type: map_at_5
value: 25.023
- type: mrr_at_1
value: 23.41417910447761
- type: mrr_at_10
value: 28.721681769722814
- type: mrr_at_100
value: 29.447477823431417
- type: mrr_at_1000
value: 29.530035280072532
- type: mrr_at_20
value: 29.070533487012906
- type: mrr_at_3
value: 27.300995024875636
- type: mrr_at_5
value: 28.005286069651742
- type: nauc_map_at_1000_diff1
value: 54.127306767400405
- type: nauc_map_at_1000_max
value: 54.299668904120466
- type: nauc_map_at_1000_std
value: 19.09102509339075
- type: nauc_map_at_100_diff1
value: 54.14930030841458
- type: nauc_map_at_100_max
value: 54.28748075070546
- type: nauc_map_at_100_std
value: 19.05551861645719
- type: nauc_map_at_10_diff1
value: 54.569420605080985
- type: nauc_map_at_10_max
value: 54.50588097529493
- type: nauc_map_at_10_std
value: 18.76623144148628
- type: nauc_map_at_1_diff1
value: 62.003697887797685
- type: nauc_map_at_1_max
value: 55.57252583521033
- type: nauc_map_at_1_std
value: 17.403174598472546
- type: nauc_map_at_20_diff1
value: 54.245875239527855
- type: nauc_map_at_20_max
value: 54.36185724029886
- type: nauc_map_at_20_std
value: 18.96042888356318
- type: nauc_map_at_3_diff1
value: 56.09768168971204
- type: nauc_map_at_3_max
value: 55.17747193329957
- type: nauc_map_at_3_std
value: 18.544666697561045
- type: nauc_map_at_5_diff1
value: 55.3157715909267
- type: nauc_map_at_5_max
value: 55.116853864932146
- type: nauc_map_at_5_std
value: 18.656695544124556
- type: nauc_mrr_at_1000_diff1
value: 53.95338877810967
- type: nauc_mrr_at_1000_max
value: 53.727399271797985
- type: nauc_mrr_at_1000_std
value: 19.392977986800037
- type: nauc_mrr_at_100_diff1
value: 53.95643035305548
- type: nauc_mrr_at_100_max
value: 53.70860178042912
- type: nauc_mrr_at_100_std
value: 19.36648647823648
- type: nauc_mrr_at_10_diff1
value: 54.30346242914457
- type: nauc_mrr_at_10_max
value: 53.941657122416565
- type: nauc_mrr_at_10_std
value: 19.21138521944097
- type: nauc_mrr_at_1_diff1
value: 61.499117283974236
- type: nauc_mrr_at_1_max
value: 56.16806734940949
- type: nauc_mrr_at_1_std
value: 18.039326320193595
- type: nauc_mrr_at_20_diff1
value: 54.0318703561127
- type: nauc_mrr_at_20_max
value: 53.80942845052438
- type: nauc_mrr_at_20_std
value: 19.333064112196382
- type: nauc_mrr_at_3_diff1
value: 55.92438984721727
- type: nauc_mrr_at_3_max
value: 54.94624111530994
- type: nauc_mrr_at_3_std
value: 19.328100352787196
- type: nauc_mrr_at_5_diff1
value: 55.03475245496786
- type: nauc_mrr_at_5_max
value: 54.63051694001982
- type: nauc_mrr_at_5_std
value: 19.290756510933484
- type: nauc_ndcg_at_1000_diff1
value: 49.738815638626185
- type: nauc_ndcg_at_1000_max
value: 52.46278871510235
- type: nauc_ndcg_at_1000_std
value: 21.34557301691874
- type: nauc_ndcg_at_100_diff1
value: 50.27814005005321
- type: nauc_ndcg_at_100_max
value: 52.13757347323408
- type: nauc_ndcg_at_100_std
value: 20.453764115388374
- type: nauc_ndcg_at_10_diff1
value: 51.5698973573366
- type: nauc_ndcg_at_10_max
value: 53.12364796865799
- type: nauc_ndcg_at_10_std
value: 19.332249322063234
- type: nauc_ndcg_at_1_diff1
value: 61.499117283974236
- type: nauc_ndcg_at_1_max
value: 56.16806734940949
- type: nauc_ndcg_at_1_std
value: 18.039326320193595
- type: nauc_ndcg_at_20_diff1
value: 50.61970694717773
- type: nauc_ndcg_at_20_max
value: 52.69764357300335
- type: nauc_ndcg_at_20_std
value: 19.947458529836123
- type: nauc_ndcg_at_3_diff1
value: 54.44038274527162
- type: nauc_ndcg_at_3_max
value: 54.99310575681763
- type: nauc_ndcg_at_3_std
value: 19.301515611981788
- type: nauc_ndcg_at_5_diff1
value: 53.15856204762418
- type: nauc_ndcg_at_5_max
value: 54.60359963167516
- type: nauc_ndcg_at_5_std
value: 19.20627190519947
- type: nauc_precision_at_1000_diff1
value: -0.27238162328398646
- type: nauc_precision_at_1000_max
value: 12.007558281870216
- type: nauc_precision_at_1000_std
value: 15.983947502391668
- type: nauc_precision_at_100_diff1
value: 20.374894140001228
- type: nauc_precision_at_100_max
value: 31.79478274148583
- type: nauc_precision_at_100_std
value: 21.985620926758333
- type: nauc_precision_at_10_diff1
value: 35.70370123656068
- type: nauc_precision_at_10_max
value: 45.7396861515553
- type: nauc_precision_at_10_std
value: 20.882061707960226
- type: nauc_precision_at_1_diff1
value: 61.499117283974236
- type: nauc_precision_at_1_max
value: 56.16806734940949
- type: nauc_precision_at_1_std
value: 18.039326320193595
- type: nauc_precision_at_20_diff1
value: 30.43237278991477
- type: nauc_precision_at_20_max
value: 42.26570740432186
- type: nauc_precision_at_20_std
value: 22.122312558988504
- type: nauc_precision_at_3_diff1
value: 45.70226000312566
- type: nauc_precision_at_3_max
value: 52.38802203777494
- type: nauc_precision_at_3_std
value: 21.26839793686077
- type: nauc_precision_at_5_diff1
value: 41.5379196029319
- type: nauc_precision_at_5_max
value: 50.48779771302531
- type: nauc_precision_at_5_std
value: 21.011023042265357
- type: nauc_recall_at_1000_diff1
value: 26.104480544558804
- type: nauc_recall_at_1000_max
value: 41.83911002304045
- type: nauc_recall_at_1000_std
value: 34.00882184767328
- type: nauc_recall_at_100_diff1
value: 36.33007260111357
- type: nauc_recall_at_100_max
value: 41.60893411064684
- type: nauc_recall_at_100_std
value: 23.149956860251038
- type: nauc_recall_at_10_diff1
value: 42.46039057055957
- type: nauc_recall_at_10_max
value: 47.91661385847294
- type: nauc_recall_at_10_std
value: 19.49980167228191
- type: nauc_recall_at_1_diff1
value: 62.003697887797685
- type: nauc_recall_at_1_max
value: 55.57252583521033
- type: nauc_recall_at_1_std
value: 17.403174598472546
- type: nauc_recall_at_20_diff1
value: 39.04564897936449
- type: nauc_recall_at_20_max
value: 46.07022230928468
- type: nauc_recall_at_20_std
value: 21.398213717295583
- type: nauc_recall_at_3_diff1
value: 50.350958384942004
- type: nauc_recall_at_3_max
value: 53.65016760703685
- type: nauc_recall_at_3_std
value: 19.63241620847436
- type: nauc_recall_at_5_diff1
value: 46.99620573974604
- type: nauc_recall_at_5_max
value: 52.542772864454825
- type: nauc_recall_at_5_std
value: 19.52551270514639
- type: ndcg_at_1
value: 23.413999999999998
- type: ndcg_at_10
value: 28.954
- type: ndcg_at_100
value: 33.155
- type: ndcg_at_1000
value: 35.876999999999995
- type: ndcg_at_20
value: 30.219
- type: ndcg_at_3
value: 26.319
- type: ndcg_at_5
value: 27.332
- type: precision_at_1
value: 23.413999999999998
- type: precision_at_10
value: 4.534
- type: precision_at_100
value: 0.736
- type: precision_at_1000
value: 0.108
- type: precision_at_20
value: 2.603
- type: precision_at_3
value: 11.567
- type: precision_at_5
value: 7.761
- type: recall_at_1
value: 20.124
- type: recall_at_10
value: 35.906
- type: recall_at_100
value: 55.445
- type: recall_at_1000
value: 74.726
- type: recall_at_20
value: 40.536
- type: recall_at_3
value: 28.409000000000002
- type: recall_at_5
value: 31.080999999999996
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWebmastersRetrieval (default)
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
split: test
type: mteb/cqadupstack-webmasters
metrics:
- type: main_score
value: 30.672
- type: map_at_1
value: 21.565
- type: map_at_10
value: 26.895999999999997
- type: map_at_100
value: 27.809
- type: map_at_1000
value: 27.971
- type: map_at_20
value: 27.365000000000002
- type: map_at_3
value: 25.215
- type: map_at_5
value: 25.997999999999998
- type: mrr_at_1
value: 25.889328063241106
- type: mrr_at_10
value: 30.983672124976458
- type: mrr_at_100
value: 31.58359812536778
- type: mrr_at_1000
value: 31.667906009715967
- type: mrr_at_20
value: 31.313283499548312
- type: mrr_at_3
value: 29.183135704874836
- type: mrr_at_5
value: 30.16139657444005
- type: nauc_map_at_1000_diff1
value: 52.08365075401896
- type: nauc_map_at_1000_max
value: 39.59033311475565
- type: nauc_map_at_1000_std
value: -4.098533602343384
- type: nauc_map_at_100_diff1
value: 52.112655307533316
- type: nauc_map_at_100_max
value: 39.56916904001941
- type: nauc_map_at_100_std
value: -4.3296060797807465
- type: nauc_map_at_10_diff1
value: 52.32253032179288
- type: nauc_map_at_10_max
value: 39.244934746722684
- type: nauc_map_at_10_std
value: -5.47023383241912
- type: nauc_map_at_1_diff1
value: 59.785047311501586
- type: nauc_map_at_1_max
value: 38.74693211411745
- type: nauc_map_at_1_std
value: -9.498322511293935
- type: nauc_map_at_20_diff1
value: 52.280947960110744
- type: nauc_map_at_20_max
value: 39.462931205098634
- type: nauc_map_at_20_std
value: -4.99687578190322
- type: nauc_map_at_3_diff1
value: 53.73124215229477
- type: nauc_map_at_3_max
value: 39.3529296279396
- type: nauc_map_at_3_std
value: -7.141215651514204
- type: nauc_map_at_5_diff1
value: 52.89955214782655
- type: nauc_map_at_5_max
value: 39.3923529172109
- type: nauc_map_at_5_std
value: -6.031665349187161
- type: nauc_mrr_at_1000_diff1
value: 52.4571973497615
- type: nauc_mrr_at_1000_max
value: 39.071132588043994
- type: nauc_mrr_at_1000_std
value: -3.187762617360252
- type: nauc_mrr_at_100_diff1
value: 52.44830786957295
- type: nauc_mrr_at_100_max
value: 39.05924934367337
- type: nauc_mrr_at_100_std
value: -3.214583696318392
- type: nauc_mrr_at_10_diff1
value: 52.55659755170272
- type: nauc_mrr_at_10_max
value: 38.8551394846449
- type: nauc_mrr_at_10_std
value: -3.7228236521378286
- type: nauc_mrr_at_1_diff1
value: 57.89499749111104
- type: nauc_mrr_at_1_max
value: 40.28043275199927
- type: nauc_mrr_at_1_std
value: -6.658819953922269
- type: nauc_mrr_at_20_diff1
value: 52.516591693271394
- type: nauc_mrr_at_20_max
value: 38.989511020341894
- type: nauc_mrr_at_20_std
value: -3.386835650231903
- type: nauc_mrr_at_3_diff1
value: 53.8172991382613
- type: nauc_mrr_at_3_max
value: 39.297087523205114
- type: nauc_mrr_at_3_std
value: -5.297135712565786
- type: nauc_mrr_at_5_diff1
value: 52.94253501286222
- type: nauc_mrr_at_5_max
value: 39.32810777352575
- type: nauc_mrr_at_5_std
value: -4.372275741909284
- type: nauc_ndcg_at_1000_diff1
value: 48.92457992420997
- type: nauc_ndcg_at_1000_max
value: 39.867112253567214
- type: nauc_ndcg_at_1000_std
value: 1.7257252877218558
- type: nauc_ndcg_at_100_diff1
value: 49.07652424310407
- type: nauc_ndcg_at_100_max
value: 39.50688638412232
- type: nauc_ndcg_at_100_std
value: 0.5390086367415701
- type: nauc_ndcg_at_10_diff1
value: 50.22431174935237
- type: nauc_ndcg_at_10_max
value: 38.06170028001974
- type: nauc_ndcg_at_10_std
value: -2.796428629914348
- type: nauc_ndcg_at_1_diff1
value: 57.89499749111104
- type: nauc_ndcg_at_1_max
value: 40.28043275199927
- type: nauc_ndcg_at_1_std
value: -6.658819953922269
- type: nauc_ndcg_at_20_diff1
value: 50.03499805848144
- type: nauc_ndcg_at_20_max
value: 38.560656797103746
- type: nauc_ndcg_at_20_std
value: -1.7766753592388655
- type: nauc_ndcg_at_3_diff1
value: 52.620526800213874
- type: nauc_ndcg_at_3_max
value: 39.54475898358006
- type: nauc_ndcg_at_3_std
value: -5.02148734438784
- type: nauc_ndcg_at_5_diff1
value: 51.53451257777897
- type: nauc_ndcg_at_5_max
value: 39.1216242106483
- type: nauc_ndcg_at_5_std
value: -3.7066985506505903
- type: nauc_precision_at_1000_diff1
value: 5.837298545275788
- type: nauc_precision_at_1000_max
value: 7.109149557131323
- type: nauc_precision_at_1000_std
value: 26.978458632673526
- type: nauc_precision_at_100_diff1
value: 10.956594398396843
- type: nauc_precision_at_100_max
value: 15.773727723670044
- type: nauc_precision_at_100_std
value: 28.75271409603496
- type: nauc_precision_at_10_diff1
value: 27.787085445923985
- type: nauc_precision_at_10_max
value: 32.21902574537277
- type: nauc_precision_at_10_std
value: 9.87019985535552
- type: nauc_precision_at_1_diff1
value: 57.89499749111104
- type: nauc_precision_at_1_max
value: 40.28043275199927
- type: nauc_precision_at_1_std
value: -6.658819953922269
- type: nauc_precision_at_20_diff1
value: 23.693124210150053
- type: nauc_precision_at_20_max
value: 28.802173174934104
- type: nauc_precision_at_20_std
value: 16.60493781182263
- type: nauc_precision_at_3_diff1
value: 40.40861587912303
- type: nauc_precision_at_3_max
value: 37.88059710150474
- type: nauc_precision_at_3_std
value: 0.023072559896230655
- type: nauc_precision_at_5_diff1
value: 37.328585420935475
- type: nauc_precision_at_5_max
value: 38.396516883621956
- type: nauc_precision_at_5_std
value: 4.191625157479985
- type: nauc_recall_at_1000_diff1
value: 30.26728754023336
- type: nauc_recall_at_1000_max
value: 39.75871820833193
- type: nauc_recall_at_1000_std
value: 29.17690545188557
- type: nauc_recall_at_100_diff1
value: 36.43333928618342
- type: nauc_recall_at_100_max
value: 37.9714686905082
- type: nauc_recall_at_100_std
value: 15.698570868772032
- type: nauc_recall_at_10_diff1
value: 42.17519201312022
- type: nauc_recall_at_10_max
value: 32.262404928460526
- type: nauc_recall_at_10_std
value: 1.0161932427796259
- type: nauc_recall_at_1_diff1
value: 59.785047311501586
- type: nauc_recall_at_1_max
value: 38.74693211411745
- type: nauc_recall_at_1_std
value: -9.498322511293935
- type: nauc_recall_at_20_diff1
value: 41.32119278251313
- type: nauc_recall_at_20_max
value: 33.64385419202765
- type: nauc_recall_at_20_std
value: 4.651365568910972
- type: nauc_recall_at_3_diff1
value: 47.63712036279662
- type: nauc_recall_at_3_max
value: 36.05476866533392
- type: nauc_recall_at_3_std
value: -5.905603351989224
- type: nauc_recall_at_5_diff1
value: 44.33822399962963
- type: nauc_recall_at_5_max
value: 35.328134224878795
- type: nauc_recall_at_5_std
value: -1.592754284514598
- type: ndcg_at_1
value: 25.889
- type: ndcg_at_10
value: 30.672
- type: ndcg_at_100
value: 34.502
- type: ndcg_at_1000
value: 37.597
- type: ndcg_at_20
value: 31.897
- type: ndcg_at_3
value: 28.231
- type: ndcg_at_5
value: 29.017
- type: precision_at_1
value: 25.889
- type: precision_at_10
value: 5.473999999999999
- type: precision_at_100
value: 1.047
- type: precision_at_1000
value: 0.17099999999999999
- type: precision_at_20
value: 3.35
- type: precision_at_3
value: 12.845999999999998
- type: precision_at_5
value: 8.696
- type: recall_at_1
value: 21.565
- type: recall_at_10
value: 37.187
- type: recall_at_100
value: 55.335
- type: recall_at_1000
value: 76.35
- type: recall_at_20
value: 41.835
- type: recall_at_3
value: 28.693
- type: recall_at_5
value: 31.823
task:
type: Retrieval
- dataset:
config: default
name: MTEB CQADupstackWordpressRetrieval (default)
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
split: test
type: mteb/cqadupstack-wordpress
metrics:
- type: main_score
value: 23.886
- type: map_at_1
value: 17.412
- type: map_at_10
value: 21.153
- type: map_at_100
value: 21.823
- type: map_at_1000
value: 21.919
- type: map_at_20
value: 21.422
- type: map_at_3
value: 19.545
- type: map_at_5
value: 20.369
- type: mrr_at_1
value: 19.038817005545287
- type: mrr_at_10
value: 22.988293284041898
- type: mrr_at_100
value: 23.611686551220696
- type: mrr_at_1000
value: 23.702948273166292
- type: mrr_at_20
value: 23.245482495481067
- type: mrr_at_3
value: 21.410967344423902
- type: mrr_at_5
value: 22.26124460874923
- type: nauc_map_at_1000_diff1
value: 45.080300854666426
- type: nauc_map_at_1000_max
value: 35.44954456916358
- type: nauc_map_at_1000_std
value: 10.842024217640219
- type: nauc_map_at_100_diff1
value: 45.05610784848168
- type: nauc_map_at_100_max
value: 35.422663641491255
- type: nauc_map_at_100_std
value: 10.792646180487116
- type: nauc_map_at_10_diff1
value: 45.35187809772025
- type: nauc_map_at_10_max
value: 35.50448169870723
- type: nauc_map_at_10_std
value: 10.679714715439507
- type: nauc_map_at_1_diff1
value: 52.98952342221014
- type: nauc_map_at_1_max
value: 37.95481938239674
- type: nauc_map_at_1_std
value: 10.83985190643407
- type: nauc_map_at_20_diff1
value: 45.279355175946996
- type: nauc_map_at_20_max
value: 35.46311871346033
- type: nauc_map_at_20_std
value: 10.453233604295017
- type: nauc_map_at_3_diff1
value: 46.29869978955396
- type: nauc_map_at_3_max
value: 34.89411399553517
- type: nauc_map_at_3_std
value: 9.59128423761114
- type: nauc_map_at_5_diff1
value: 45.34523174189765
- type: nauc_map_at_5_max
value: 34.9932542303531
- type: nauc_map_at_5_std
value: 10.26993355640661
- type: nauc_mrr_at_1000_diff1
value: 45.87358617605449
- type: nauc_mrr_at_1000_max
value: 36.531281344082565
- type: nauc_mrr_at_1000_std
value: 11.935019875392866
- type: nauc_mrr_at_100_diff1
value: 45.8529648088822
- type: nauc_mrr_at_100_max
value: 36.519987627643616
- type: nauc_mrr_at_100_std
value: 11.91415516629493
- type: nauc_mrr_at_10_diff1
value: 46.2267866211129
- type: nauc_mrr_at_10_max
value: 36.676897457589114
- type: nauc_mrr_at_10_std
value: 11.945697394416879
- type: nauc_mrr_at_1_diff1
value: 54.052048314939285
- type: nauc_mrr_at_1_max
value: 38.24843998104176
- type: nauc_mrr_at_1_std
value: 11.32347443333491
- type: nauc_mrr_at_20_diff1
value: 46.10342155269118
- type: nauc_mrr_at_20_max
value: 36.66310260555186
- type: nauc_mrr_at_20_std
value: 11.744940404616644
- type: nauc_mrr_at_3_diff1
value: 47.26993703583176
- type: nauc_mrr_at_3_max
value: 36.22953356126758
- type: nauc_mrr_at_3_std
value: 11.069659860675788
- type: nauc_mrr_at_5_diff1
value: 46.13993512035234
- type: nauc_mrr_at_5_max
value: 36.44937686326434
- type: nauc_mrr_at_5_std
value: 11.882323169087938
- type: nauc_ndcg_at_1000_diff1
value: 41.68272787784497
- type: nauc_ndcg_at_1000_max
value: 35.36857525462172
- type: nauc_ndcg_at_1000_std
value: 12.934491520811026
- type: nauc_ndcg_at_100_diff1
value: 41.20687111980092
- type: nauc_ndcg_at_100_max
value: 34.99414337354182
- type: nauc_ndcg_at_100_std
value: 12.613932731373154
- type: nauc_ndcg_at_10_diff1
value: 43.073788147544164
- type: nauc_ndcg_at_10_max
value: 35.54880926579014
- type: nauc_ndcg_at_10_std
value: 11.496255982990554
- type: nauc_ndcg_at_1_diff1
value: 54.052048314939285
- type: nauc_ndcg_at_1_max
value: 38.24843998104176
- type: nauc_ndcg_at_1_std
value: 11.32347443333491
- type: nauc_ndcg_at_20_diff1
value: 42.83861403717052
- type: nauc_ndcg_at_20_max
value: 35.37409034798095
- type: nauc_ndcg_at_20_std
value: 10.725759893085879
- type: nauc_ndcg_at_3_diff1
value: 44.614319740276606
- type: nauc_ndcg_at_3_max
value: 34.37947584951607
- type: nauc_ndcg_at_3_std
value: 9.582763290394409
- type: nauc_ndcg_at_5_diff1
value: 42.74393042799707
- type: nauc_ndcg_at_5_max
value: 34.56318914727522
- type: nauc_ndcg_at_5_std
value: 10.791463962408246
- type: nauc_precision_at_1000_diff1
value: 19.360640317938135
- type: nauc_precision_at_1000_max
value: 23.598410647832544
- type: nauc_precision_at_1000_std
value: 20.068230147834342
- type: nauc_precision_at_100_diff1
value: 27.583431814893217
- type: nauc_precision_at_100_max
value: 32.0373769441221
- type: nauc_precision_at_100_std
value: 21.840961317913457
- type: nauc_precision_at_10_diff1
value: 37.17543628652202
- type: nauc_precision_at_10_max
value: 36.13471233966831
- type: nauc_precision_at_10_std
value: 15.187885199748804
- type: nauc_precision_at_1_diff1
value: 54.052048314939285
- type: nauc_precision_at_1_max
value: 38.24843998104176
- type: nauc_precision_at_1_std
value: 11.32347443333491
- type: nauc_precision_at_20_diff1
value: 36.313693961663354
- type: nauc_precision_at_20_max
value: 34.98349203145928
- type: nauc_precision_at_20_std
value: 12.1991019320839
- type: nauc_precision_at_3_diff1
value: 38.90231899476045
- type: nauc_precision_at_3_max
value: 32.492833274823134
- type: nauc_precision_at_3_std
value: 9.735527613731456
- type: nauc_precision_at_5_diff1
value: 35.94202366490917
- type: nauc_precision_at_5_max
value: 32.89733460949656
- type: nauc_precision_at_5_std
value: 12.260002599166384
- type: nauc_recall_at_1000_diff1
value: 25.168921326020754
- type: nauc_recall_at_1000_max
value: 31.686605591498022
- type: nauc_recall_at_1000_std
value: 19.03561399081122
- type: nauc_recall_at_100_diff1
value: 27.026856793111957
- type: nauc_recall_at_100_max
value: 30.976614077143722
- type: nauc_recall_at_100_std
value: 17.226971695826464
- type: nauc_recall_at_10_diff1
value: 35.62607649761852
- type: nauc_recall_at_10_max
value: 33.7501283641428
- type: nauc_recall_at_10_std
value: 12.285657222493427
- type: nauc_recall_at_1_diff1
value: 52.98952342221014
- type: nauc_recall_at_1_max
value: 37.95481938239674
- type: nauc_recall_at_1_std
value: 10.83985190643407
- type: nauc_recall_at_20_diff1
value: 34.83633695309472
- type: nauc_recall_at_20_max
value: 32.86771508007259
- type: nauc_recall_at_20_std
value: 9.678829364636606
- type: nauc_recall_at_3_diff1
value: 38.03697585235963
- type: nauc_recall_at_3_max
value: 30.476017013445805
- type: nauc_recall_at_3_std
value: 7.567446018559129
- type: nauc_recall_at_5_diff1
value: 34.62516359445759
- type: nauc_recall_at_5_max
value: 31.653312238841597
- type: nauc_recall_at_5_std
value: 10.731712405384886
- type: ndcg_at_1
value: 19.039
- type: ndcg_at_10
value: 23.886
- type: ndcg_at_100
value: 27.744999999999997
- type: ndcg_at_1000
value: 30.446
- type: ndcg_at_20
value: 24.809
- type: ndcg_at_3
value: 20.723
- type: ndcg_at_5
value: 22.146
- type: precision_at_1
value: 19.039
- type: precision_at_10
value: 3.567
- type: precision_at_100
value: 0.5910000000000001
- type: precision_at_1000
value: 0.08800000000000001
- type: precision_at_20
value: 1.9869999999999999
- type: precision_at_3
value: 8.38
- type: precision_at_5
value: 5.804
- type: recall_at_1
value: 17.412
- type: recall_at_10
value: 30.604
- type: recall_at_100
value: 49.494
- type: recall_at_1000
value: 70.172
- type: recall_at_20
value: 34.054
- type: recall_at_3
value: 22.286
- type: recall_at_5
value: 25.705
task:
type: Retrieval
- dataset:
config: default
name: MTEB ClimateFEVER (default)
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
split: test
type: mteb/climate-fever
metrics:
- type: main_score
value: 28.212
- type: map_at_1
value: 13.017000000000001
- type: map_at_10
value: 20.321
- type: map_at_100
value: 21.604
- type: map_at_1000
value: 21.755
- type: map_at_20
value: 21.032
- type: map_at_3
value: 17.49
- type: map_at_5
value: 19.057
- type: mrr_at_1
value: 30.09771986970684
- type: mrr_at_10
value: 39.847319166537396
- type: mrr_at_100
value: 40.690633269363104
- type: mrr_at_1000
value: 40.7338784607264
- type: mrr_at_20
value: 40.37801359085934
- type: mrr_at_3
value: 36.89467969598261
- type: mrr_at_5
value: 38.59500542888165
- type: nauc_map_at_1000_diff1
value: 28.9604375397966
- type: nauc_map_at_1000_max
value: 32.35383323725253
- type: nauc_map_at_1000_std
value: 25.268075508181926
- type: nauc_map_at_100_diff1
value: 28.91401321002612
- type: nauc_map_at_100_max
value: 32.335595208992984
- type: nauc_map_at_100_std
value: 25.154330042862664
- type: nauc_map_at_10_diff1
value: 28.656813940382115
- type: nauc_map_at_10_max
value: 31.365964173994556
- type: nauc_map_at_10_std
value: 23.39762822495535
- type: nauc_map_at_1_diff1
value: 34.48035472714226
- type: nauc_map_at_1_max
value: 28.934161583551422
- type: nauc_map_at_1_std
value: 18.741105759186734
- type: nauc_map_at_20_diff1
value: 28.69991210450139
- type: nauc_map_at_20_max
value: 31.87878353118675
- type: nauc_map_at_20_std
value: 24.301765643370874
- type: nauc_map_at_3_diff1
value: 29.96927690741529
- type: nauc_map_at_3_max
value: 30.45113949685312
- type: nauc_map_at_3_std
value: 21.37090419963093
- type: nauc_map_at_5_diff1
value: 28.991910366109906
- type: nauc_map_at_5_max
value: 30.81393946836492
- type: nauc_map_at_5_std
value: 22.5204803456172
- type: nauc_mrr_at_1000_diff1
value: 32.68111770309144
- type: nauc_mrr_at_1000_max
value: 34.934889264530625
- type: nauc_mrr_at_1000_std
value: 27.704825255836802
- type: nauc_mrr_at_100_diff1
value: 32.6710673829821
- type: nauc_mrr_at_100_max
value: 34.9393131792049
- type: nauc_mrr_at_100_std
value: 27.718374360852096
- type: nauc_mrr_at_10_diff1
value: 32.67458190839553
- type: nauc_mrr_at_10_max
value: 34.953542924313886
- type: nauc_mrr_at_10_std
value: 27.56927045791106
- type: nauc_mrr_at_1_diff1
value: 35.47417523435113
- type: nauc_mrr_at_1_max
value: 33.54020583678914
- type: nauc_mrr_at_1_std
value: 24.229131075911837
- type: nauc_mrr_at_20_diff1
value: 32.58996187437461
- type: nauc_mrr_at_20_max
value: 34.92896642367715
- type: nauc_mrr_at_20_std
value: 27.695025910282123
- type: nauc_mrr_at_3_diff1
value: 32.96446609776662
- type: nauc_mrr_at_3_max
value: 34.46013252103825
- type: nauc_mrr_at_3_std
value: 26.247102509795937
- type: nauc_mrr_at_5_diff1
value: 32.58237165372756
- type: nauc_mrr_at_5_max
value: 34.740209335604874
- type: nauc_mrr_at_5_std
value: 26.831783830208312
- type: nauc_ndcg_at_1000_diff1
value: 30.014944999905314
- type: nauc_ndcg_at_1000_max
value: 35.843285734507724
- type: nauc_ndcg_at_1000_std
value: 32.79499358419293
- type: nauc_ndcg_at_100_diff1
value: 29.1269981224563
- type: nauc_ndcg_at_100_max
value: 35.57247072879993
- type: nauc_ndcg_at_100_std
value: 31.441880943622397
- type: nauc_ndcg_at_10_diff1
value: 28.524256725934876
- type: nauc_ndcg_at_10_max
value: 33.12204032386719
- type: nauc_ndcg_at_10_std
value: 26.447786896078334
- type: nauc_ndcg_at_1_diff1
value: 35.47417523435113
- type: nauc_ndcg_at_1_max
value: 33.54020583678914
- type: nauc_ndcg_at_1_std
value: 24.229131075911837
- type: nauc_ndcg_at_20_diff1
value: 28.463383747538874
- type: nauc_ndcg_at_20_max
value: 34.066601998384414
- type: nauc_ndcg_at_20_std
value: 28.36971182992064
- type: nauc_ndcg_at_3_diff1
value: 29.833077492596793
- type: nauc_ndcg_at_3_max
value: 31.90747339897323
- type: nauc_ndcg_at_3_std
value: 23.741064443521083
- type: nauc_ndcg_at_5_diff1
value: 28.673932316507095
- type: nauc_ndcg_at_5_max
value: 32.31146189714115
- type: nauc_ndcg_at_5_std
value: 24.748445626346818
- type: nauc_precision_at_1000_diff1
value: 13.101981588417978
- type: nauc_precision_at_1000_max
value: 18.980434446047852
- type: nauc_precision_at_1000_std
value: 34.894676080745214
- type: nauc_precision_at_100_diff1
value: 15.977351454837574
- type: nauc_precision_at_100_max
value: 28.27449565578445
- type: nauc_precision_at_100_std
value: 38.953946289000164
- type: nauc_precision_at_10_diff1
value: 19.76477192741852
- type: nauc_precision_at_10_max
value: 30.578736423173066
- type: nauc_precision_at_10_std
value: 31.120926070901795
- type: nauc_precision_at_1_diff1
value: 35.47417523435113
- type: nauc_precision_at_1_max
value: 33.54020583678914
- type: nauc_precision_at_1_std
value: 24.229131075911837
- type: nauc_precision_at_20_diff1
value: 18.01695906942538
- type: nauc_precision_at_20_max
value: 30.625106221982996
- type: nauc_precision_at_20_std
value: 34.398947804377464
- type: nauc_precision_at_3_diff1
value: 25.05803533689324
- type: nauc_precision_at_3_max
value: 31.971009197662305
- type: nauc_precision_at_3_std
value: 26.97185456877385
- type: nauc_precision_at_5_diff1
value: 21.285339148661627
- type: nauc_precision_at_5_max
value: 30.951521247982217
- type: nauc_precision_at_5_std
value: 28.845266743639424
- type: nauc_recall_at_1000_diff1
value: 21.666236554672572
- type: nauc_recall_at_1000_max
value: 32.90881844420523
- type: nauc_recall_at_1000_std
value: 43.25106650136571
- type: nauc_recall_at_100_diff1
value: 18.79493296793092
- type: nauc_recall_at_100_max
value: 32.04590263165258
- type: nauc_recall_at_100_std
value: 35.4517527204907
- type: nauc_recall_at_10_diff1
value: 20.679017825223646
- type: nauc_recall_at_10_max
value: 28.345295191030146
- type: nauc_recall_at_10_std
value: 24.062716299883178
- type: nauc_recall_at_1_diff1
value: 34.48035472714226
- type: nauc_recall_at_1_max
value: 28.934161583551422
- type: nauc_recall_at_1_std
value: 18.741105759186734
- type: nauc_recall_at_20_diff1
value: 19.492610813475135
- type: nauc_recall_at_20_max
value: 29.71354758230516
- type: nauc_recall_at_20_std
value: 27.83429445647274
- type: nauc_recall_at_3_diff1
value: 25.377114884402918
- type: nauc_recall_at_3_max
value: 29.27911716127374
- type: nauc_recall_at_3_std
value: 21.284373865740655
- type: nauc_recall_at_5_diff1
value: 22.384806106738502
- type: nauc_recall_at_5_max
value: 28.802262502032335
- type: nauc_recall_at_5_std
value: 22.351166917570502
- type: ndcg_at_1
value: 30.098000000000003
- type: ndcg_at_10
value: 28.212
- type: ndcg_at_100
value: 33.597
- type: ndcg_at_1000
value: 36.581
- type: ndcg_at_20
value: 30.342999999999996
- type: ndcg_at_3
value: 23.995
- type: ndcg_at_5
value: 25.442999999999998
- type: precision_at_1
value: 30.098000000000003
- type: precision_at_10
value: 8.365
- type: precision_at_100
value: 1.415
- type: precision_at_1000
value: 0.197
- type: precision_at_20
value: 5.098
- type: precision_at_3
value: 17.459
- type: precision_at_5
value: 13.147
- type: recall_at_1
value: 13.017000000000001
- type: recall_at_10
value: 32.138
- type: recall_at_100
value: 50.739999999999995
- type: recall_at_1000
value: 67.605
- type: recall_at_20
value: 38.135000000000005
- type: recall_at_3
value: 21.156
- type: recall_at_5
value: 25.945
task:
type: Retrieval
- dataset:
config: default
name: MTEB CmedqaRetrieval (default)
revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
split: test
type: C-MTEB/CmedqaRetrieval
metrics:
- type: main_score
value: 22.649
- type: map_at_1
value: 12.894
- type: map_at_10
value: 18.398
- type: map_at_100
value: 19.459
- type: map_at_1000
value: 19.582
- type: map_at_20
value: 18.959
- type: map_at_3
value: 16.463
- type: map_at_5
value: 17.378
- type: mrr_at_1
value: 20.855213803450862
- type: mrr_at_10
value: 25.80732484708481
- type: mrr_at_100
value: 26.581761181113624
- type: mrr_at_1000
value: 26.656399615067492
- type: mrr_at_20
value: 26.247195370637264
- type: mrr_at_3
value: 24.122697341001953
- type: mrr_at_5
value: 24.93289989163964
- type: nauc_map_at_1000_diff1
value: 43.84274599218202
- type: nauc_map_at_1000_max
value: 45.479276120298344
- type: nauc_map_at_1000_std
value: 12.122621024712403
- type: nauc_map_at_100_diff1
value: 43.84890191966983
- type: nauc_map_at_100_max
value: 45.43765669444242
- type: nauc_map_at_100_std
value: 11.966671645957028
- type: nauc_map_at_10_diff1
value: 44.36458226656199
- type: nauc_map_at_10_max
value: 45.198112497326164
- type: nauc_map_at_10_std
value: 11.031085313048038
- type: nauc_map_at_1_diff1
value: 50.77021361612485
- type: nauc_map_at_1_max
value: 44.9428775206887
- type: nauc_map_at_1_std
value: 9.060261883539706
- type: nauc_map_at_20_diff1
value: 44.07397948013123
- type: nauc_map_at_20_max
value: 45.273848594038455
- type: nauc_map_at_20_std
value: 11.330853124721832
- type: nauc_map_at_3_diff1
value: 46.12522696117035
- type: nauc_map_at_3_max
value: 45.29688612734072
- type: nauc_map_at_3_std
value: 10.419110119634555
- type: nauc_map_at_5_diff1
value: 44.98363487026353
- type: nauc_map_at_5_max
value: 45.37700576168934
- type: nauc_map_at_5_std
value: 10.898739429249941
- type: nauc_mrr_at_1000_diff1
value: 46.292701489536434
- type: nauc_mrr_at_1000_max
value: 47.43736463676237
- type: nauc_mrr_at_1000_std
value: 12.830971267129515
- type: nauc_mrr_at_100_diff1
value: 46.28459872281162
- type: nauc_mrr_at_100_max
value: 47.428218400316375
- type: nauc_mrr_at_100_std
value: 12.788554254151135
- type: nauc_mrr_at_10_diff1
value: 46.60698297272493
- type: nauc_mrr_at_10_max
value: 47.49916150124582
- type: nauc_mrr_at_10_std
value: 12.318695960203996
- type: nauc_mrr_at_1_diff1
value: 52.474628752419996
- type: nauc_mrr_at_1_max
value: 49.925997446056876
- type: nauc_mrr_at_1_std
value: 11.922730379304515
- type: nauc_mrr_at_20_diff1
value: 46.40749343170004
- type: nauc_mrr_at_20_max
value: 47.42151132322358
- type: nauc_mrr_at_20_std
value: 12.541667676745117
- type: nauc_mrr_at_3_diff1
value: 48.18192695522407
- type: nauc_mrr_at_3_max
value: 48.26399499335713
- type: nauc_mrr_at_3_std
value: 12.024058074658857
- type: nauc_mrr_at_5_diff1
value: 47.28360755810589
- type: nauc_mrr_at_5_max
value: 47.9793603282811
- type: nauc_mrr_at_5_std
value: 12.29211337620231
- type: nauc_ndcg_at_1000_diff1
value: 40.26744863332448
- type: nauc_ndcg_at_1000_max
value: 45.61725612901479
- type: nauc_ndcg_at_1000_std
value: 18.052160309983826
- type: nauc_ndcg_at_100_diff1
value: 40.29735516818454
- type: nauc_ndcg_at_100_max
value: 45.08776191822656
- type: nauc_ndcg_at_100_std
value: 15.760044344733448
- type: nauc_ndcg_at_10_diff1
value: 42.25294505119997
- type: nauc_ndcg_at_10_max
value: 44.699284633767036
- type: nauc_ndcg_at_10_std
value: 11.747981699271111
- type: nauc_ndcg_at_1_diff1
value: 52.474628752419996
- type: nauc_ndcg_at_1_max
value: 49.925997446056876
- type: nauc_ndcg_at_1_std
value: 11.922730379304515
- type: nauc_ndcg_at_20_diff1
value: 41.394427654655445
- type: nauc_ndcg_at_20_max
value: 44.48687052147322
- type: nauc_ndcg_at_20_std
value: 12.416604099649925
- type: nauc_ndcg_at_3_diff1
value: 45.49543977900659
- type: nauc_ndcg_at_3_max
value: 46.87132909772721
- type: nauc_ndcg_at_3_std
value: 11.870163932604662
- type: nauc_ndcg_at_5_diff1
value: 43.70667389695487
- type: nauc_ndcg_at_5_max
value: 45.979785032074275
- type: nauc_ndcg_at_5_std
value: 12.006387457042322
- type: nauc_precision_at_1000_diff1
value: 11.611809966547701
- type: nauc_precision_at_1000_max
value: 27.978399208595267
- type: nauc_precision_at_1000_std
value: 29.153711078952217
- type: nauc_precision_at_100_diff1
value: 18.532317578553656
- type: nauc_precision_at_100_max
value: 34.81158574637404
- type: nauc_precision_at_100_std
value: 25.374246665283945
- type: nauc_precision_at_10_diff1
value: 30.90718618161642
- type: nauc_precision_at_10_max
value: 42.47485452969173
- type: nauc_precision_at_10_std
value: 15.251594945393599
- type: nauc_precision_at_1_diff1
value: 52.474628752419996
- type: nauc_precision_at_1_max
value: 49.925997446056876
- type: nauc_precision_at_1_std
value: 11.922730379304515
- type: nauc_precision_at_20_diff1
value: 26.406002658570642
- type: nauc_precision_at_20_max
value: 39.884817225812135
- type: nauc_precision_at_20_std
value: 17.31215639396359
- type: nauc_precision_at_3_diff1
value: 40.42388812791282
- type: nauc_precision_at_3_max
value: 47.64175858827733
- type: nauc_precision_at_3_std
value: 13.853162207213806
- type: nauc_precision_at_5_diff1
value: 35.70789816053672
- type: nauc_precision_at_5_max
value: 46.61093476897344
- type: nauc_precision_at_5_std
value: 15.525449340593584
- type: nauc_recall_at_1000_diff1
value: 18.837729900819607
- type: nauc_recall_at_1000_max
value: 37.25596617664244
- type: nauc_recall_at_1000_std
value: 41.12107733667324
- type: nauc_recall_at_100_diff1
value: 23.115561312978162
- type: nauc_recall_at_100_max
value: 35.71918283879486
- type: nauc_recall_at_100_std
value: 23.989941519505628
- type: nauc_recall_at_10_diff1
value: 31.59189289756229
- type: nauc_recall_at_10_max
value: 36.38716126126731
- type: nauc_recall_at_10_std
value: 10.495364039694593
- type: nauc_recall_at_1_diff1
value: 50.77021361612485
- type: nauc_recall_at_1_max
value: 44.9428775206887
- type: nauc_recall_at_1_std
value: 9.060261883539706
- type: nauc_recall_at_20_diff1
value: 28.830785779330366
- type: nauc_recall_at_20_max
value: 35.393348139603475
- type: nauc_recall_at_20_std
value: 12.102999068196619
- type: nauc_recall_at_3_diff1
value: 40.542520678382644
- type: nauc_recall_at_3_max
value: 41.915008181994324
- type: nauc_recall_at_3_std
value: 10.420365070614944
- type: nauc_recall_at_5_diff1
value: 36.55947416696159
- type: nauc_recall_at_5_max
value: 40.544255671331875
- type: nauc_recall_at_5_std
value: 11.190415699822086
- type: ndcg_at_1
value: 20.855
- type: ndcg_at_10
value: 22.649
- type: ndcg_at_100
value: 27.46
- type: ndcg_at_1000
value: 30.311
- type: ndcg_at_20
value: 24.356
- type: ndcg_at_3
value: 19.908
- type: ndcg_at_5
value: 20.686
- type: precision_at_1
value: 20.855
- type: precision_at_10
value: 5.116
- type: precision_at_100
value: 0.914
- type: precision_at_1000
value: 0.129
- type: precision_at_20
value: 3.147
- type: precision_at_3
value: 11.294
- type: precision_at_5
value: 8.022
- type: recall_at_1
value: 12.894
- type: recall_at_10
value: 28.249000000000002
- type: recall_at_100
value: 48.44
- type: recall_at_1000
value: 68.184
- type: recall_at_20
value: 33.973
- type: recall_at_3
value: 19.381
- type: recall_at_5
value: 22.358
task:
type: Retrieval
- dataset:
config: default
name: MTEB CovidRetrieval (default)
revision: 1271c7809071a13532e05f25fb53511ffce77117
split: test
type: C-MTEB/CovidRetrieval
metrics:
- type: main_score
value: 85.541
- type: map_at_1
value: 76.712
- type: map_at_10
value: 82.76599999999999
- type: map_at_100
value: 82.95400000000001
- type: map_at_1000
value: 82.95700000000001
- type: map_at_20
value: 82.899
- type: map_at_3
value: 81.48100000000001
- type: map_at_5
value: 82.26599999999999
- type: mrr_at_1
value: 76.92307692307693
- type: mrr_at_10
value: 82.80349407061732
- type: mrr_at_100
value: 82.98207452799001
- type: mrr_at_1000
value: 82.9857038248515
- type: mrr_at_20
value: 82.92710961408577
- type: mrr_at_3
value: 81.61222339304531
- type: mrr_at_5
value: 82.36037934668072
- type: nauc_map_at_1000_diff1
value: 83.6771226005355
- type: nauc_map_at_1000_max
value: 52.427797521096885
- type: nauc_map_at_1000_std
value: -22.46855714516629
- type: nauc_map_at_100_diff1
value: 83.67554898856211
- type: nauc_map_at_100_max
value: 52.43667308693426
- type: nauc_map_at_100_std
value: -22.453092384080296
- type: nauc_map_at_10_diff1
value: 83.63377660675229
- type: nauc_map_at_10_max
value: 52.507092302621814
- type: nauc_map_at_10_std
value: -22.599466162417798
- type: nauc_map_at_1_diff1
value: 86.15059546844765
- type: nauc_map_at_1_max
value: 49.70612292973377
- type: nauc_map_at_1_std
value: -24.574205863138655
- type: nauc_map_at_20_diff1
value: 83.6571122442771
- type: nauc_map_at_20_max
value: 52.53003487116206
- type: nauc_map_at_20_std
value: -22.29199493949789
- type: nauc_map_at_3_diff1
value: 83.53600182926077
- type: nauc_map_at_3_max
value: 51.48081639152428
- type: nauc_map_at_3_std
value: -23.95292363507194
- type: nauc_map_at_5_diff1
value: 83.71395002281027
- type: nauc_map_at_5_max
value: 52.37004278078251
- type: nauc_map_at_5_std
value: -23.366003089734853
- type: nauc_mrr_at_1000_diff1
value: 83.61995524968707
- type: nauc_mrr_at_1000_max
value: 52.477542458550886
- type: nauc_mrr_at_1000_std
value: -22.333080741545132
- type: nauc_mrr_at_100_diff1
value: 83.61839299882983
- type: nauc_mrr_at_100_max
value: 52.486416698332306
- type: nauc_mrr_at_100_std
value: -22.31762952128872
- type: nauc_mrr_at_10_diff1
value: 83.57540058737065
- type: nauc_mrr_at_10_max
value: 52.545507004997816
- type: nauc_mrr_at_10_std
value: -22.46871032971557
- type: nauc_mrr_at_1_diff1
value: 85.83991138913477
- type: nauc_mrr_at_1_max
value: 49.57990159528055
- type: nauc_mrr_at_1_std
value: -24.43162367994886
- type: nauc_mrr_at_20_diff1
value: 83.60013381041259
- type: nauc_mrr_at_20_max
value: 52.57971181654748
- type: nauc_mrr_at_20_std
value: -22.156819477173133
- type: nauc_mrr_at_3_diff1
value: 83.30267568781416
- type: nauc_mrr_at_3_max
value: 51.60809167890664
- type: nauc_mrr_at_3_std
value: -23.473315407718783
- type: nauc_mrr_at_5_diff1
value: 83.55906056304644
- type: nauc_mrr_at_5_max
value: 52.40224088006019
- type: nauc_mrr_at_5_std
value: -23.08319811392415
- type: nauc_ndcg_at_1000_diff1
value: 83.03284671941663
- type: nauc_ndcg_at_1000_max
value: 53.39643002819807
- type: nauc_ndcg_at_1000_std
value: -21.12492526587505
- type: nauc_ndcg_at_100_diff1
value: 83.0125296344548
- type: nauc_ndcg_at_100_max
value: 53.803302728432875
- type: nauc_ndcg_at_100_std
value: -20.433612936956884
- type: nauc_ndcg_at_10_diff1
value: 82.75722949475718
- type: nauc_ndcg_at_10_max
value: 54.52418873184555
- type: nauc_ndcg_at_10_std
value: -20.383704043222163
- type: nauc_ndcg_at_1_diff1
value: 85.83991138913477
- type: nauc_ndcg_at_1_max
value: 49.57990159528055
- type: nauc_ndcg_at_1_std
value: -24.43162367994886
- type: nauc_ndcg_at_20_diff1
value: 82.82405119235945
- type: nauc_ndcg_at_20_max
value: 54.674440378558366
- type: nauc_ndcg_at_20_std
value: -19.008326210636753
- type: nauc_ndcg_at_3_diff1
value: 82.46076801673205
- type: nauc_ndcg_at_3_max
value: 52.26282641124796
- type: nauc_ndcg_at_3_std
value: -23.26603157004372
- type: nauc_ndcg_at_5_diff1
value: 82.83681106420133
- type: nauc_ndcg_at_5_max
value: 54.07223605483471
- type: nauc_ndcg_at_5_std
value: -22.219817031191482
- type: nauc_precision_at_1000_diff1
value: -37.23223858169585
- type: nauc_precision_at_1000_max
value: -11.169580444647227
- type: nauc_precision_at_1000_std
value: 21.88031832418344
- type: nauc_precision_at_100_diff1
value: 3.301752389000813
- type: nauc_precision_at_100_max
value: 23.553402973772553
- type: nauc_precision_at_100_std
value: 28.91056452755429
- type: nauc_precision_at_10_diff1
value: 47.76680402893545
- type: nauc_precision_at_10_max
value: 50.02506814703298
- type: nauc_precision_at_10_std
value: 5.423151633085648
- type: nauc_precision_at_1_diff1
value: 85.83991138913477
- type: nauc_precision_at_1_max
value: 49.57990159528055
- type: nauc_precision_at_1_std
value: -24.43162367994886
- type: nauc_precision_at_20_diff1
value: 34.78818210434864
- type: nauc_precision_at_20_max
value: 47.13251460889552
- type: nauc_precision_at_20_std
value: 25.00716401931335
- type: nauc_precision_at_3_diff1
value: 72.13849414907274
- type: nauc_precision_at_3_max
value: 51.24622711077072
- type: nauc_precision_at_3_std
value: -19.119810033196142
- type: nauc_precision_at_5_diff1
value: 66.19703459795305
- type: nauc_precision_at_5_max
value: 55.21530135525487
- type: nauc_precision_at_5_std
value: -12.072221200376907
- type: nauc_recall_at_1000_diff1
value: 40.77748260957538
- type: nauc_recall_at_1000_max
value: 55.770307506563356
- type: nauc_recall_at_1000_std
value: -14.057009165085931
- type: nauc_recall_at_100_diff1
value: 71.07032398970254
- type: nauc_recall_at_100_max
value: 81.63691298199028
- type: nauc_recall_at_100_std
value: 27.473092342222866
- type: nauc_recall_at_10_diff1
value: 76.77520256165532
- type: nauc_recall_at_10_max
value: 70.96790722488872
- type: nauc_recall_at_10_std
value: -2.2469959080331305
- type: nauc_recall_at_1_diff1
value: 86.15059546844765
- type: nauc_recall_at_1_max
value: 49.70612292973377
- type: nauc_recall_at_1_std
value: -24.574205863138655
- type: nauc_recall_at_20_diff1
value: 74.88483995648564
- type: nauc_recall_at_20_max
value: 79.9802787833889
- type: nauc_recall_at_20_std
value: 22.714752655241625
- type: nauc_recall_at_3_diff1
value: 78.44674446588604
- type: nauc_recall_at_3_max
value: 55.19181785928796
- type: nauc_recall_at_3_std
value: -21.150826693512215
- type: nauc_recall_at_5_diff1
value: 78.74767212963312
- type: nauc_recall_at_5_max
value: 63.0860506176757
- type: nauc_recall_at_5_std
value: -16.69010257929412
- type: ndcg_at_1
value: 76.923
- type: ndcg_at_10
value: 85.541
- type: ndcg_at_100
value: 86.461
- type: ndcg_at_1000
value: 86.598
- type: ndcg_at_20
value: 86.00999999999999
- type: ndcg_at_3
value: 83.048
- type: ndcg_at_5
value: 84.435
- type: precision_at_1
value: 76.923
- type: precision_at_10
value: 9.504999999999999
- type: precision_at_100
value: 0.9950000000000001
- type: precision_at_1000
value: 0.101
- type: precision_at_20
value: 4.8469999999999995
- type: precision_at_3
value: 29.329
- type: precision_at_5
value: 18.293
- type: recall_at_1
value: 76.712
- type: recall_at_10
value: 94.04599999999999
- type: recall_at_100
value: 98.419
- type: recall_at_1000
value: 99.579
- type: recall_at_20
value: 95.89
- type: recall_at_3
value: 87.434
- type: recall_at_5
value: 90.78
task:
type: Retrieval
- dataset:
config: default
name: MTEB DBPedia (default)
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
split: test
type: mteb/dbpedia
metrics:
- type: main_score
value: 20.819
- type: map_at_1
value: 3.628
- type: map_at_10
value: 8.857
- type: map_at_100
value: 12.588
- type: map_at_1000
value: 13.258000000000001
- type: map_at_20
value: 10.276
- type: map_at_3
value: 6.392
- type: map_at_5
value: 7.53
- type: mrr_at_1
value: 34.25
- type: mrr_at_10
value: 43.496825396825386
- type: mrr_at_100
value: 44.12908157180389
- type: mrr_at_1000
value: 44.15803926353856
- type: mrr_at_20
value: 43.88608249872761
- type: mrr_at_3
value: 41.5
- type: mrr_at_5
value: 42.9625
- type: nauc_map_at_1000_diff1
value: 37.37394986476202
- type: nauc_map_at_1000_max
value: 59.345239146520754
- type: nauc_map_at_1000_std
value: 55.169697546679295
- type: nauc_map_at_100_diff1
value: 37.6559779781504
- type: nauc_map_at_100_max
value: 58.942087337068585
- type: nauc_map_at_100_std
value: 54.045690215259675
- type: nauc_map_at_10_diff1
value: 39.09378782193848
- type: nauc_map_at_10_max
value: 53.55846019231183
- type: nauc_map_at_10_std
value: 41.630710620488465
- type: nauc_map_at_1_diff1
value: 43.38566026898597
- type: nauc_map_at_1_max
value: 37.14628586578234
- type: nauc_map_at_1_std
value: 26.2042447918162
- type: nauc_map_at_20_diff1
value: 39.00481968723579
- type: nauc_map_at_20_max
value: 56.04031067878328
- type: nauc_map_at_20_std
value: 45.76425742666721
- type: nauc_map_at_3_diff1
value: 37.322521097626705
- type: nauc_map_at_3_max
value: 46.04542793269727
- type: nauc_map_at_3_std
value: 33.3952162038131
- type: nauc_map_at_5_diff1
value: 38.97015740798845
- type: nauc_map_at_5_max
value: 49.89258673855368
- type: nauc_map_at_5_std
value: 37.22020377839247
- type: nauc_mrr_at_1000_diff1
value: 39.83045919993469
- type: nauc_mrr_at_1000_max
value: 50.726537573809985
- type: nauc_mrr_at_1000_std
value: 50.0130436962735
- type: nauc_mrr_at_100_diff1
value: 39.81822425813205
- type: nauc_mrr_at_100_max
value: 50.740733722342966
- type: nauc_mrr_at_100_std
value: 50.01889514657749
- type: nauc_mrr_at_10_diff1
value: 40.13805398263145
- type: nauc_mrr_at_10_max
value: 50.6618423067005
- type: nauc_mrr_at_10_std
value: 49.76359877028564
- type: nauc_mrr_at_1_diff1
value: 43.60342597401739
- type: nauc_mrr_at_1_max
value: 46.72967688790819
- type: nauc_mrr_at_1_std
value: 46.08259257176318
- type: nauc_mrr_at_20_diff1
value: 39.89191042838717
- type: nauc_mrr_at_20_max
value: 50.70620407117266
- type: nauc_mrr_at_20_std
value: 49.929325494823686
- type: nauc_mrr_at_3_diff1
value: 39.181127533069606
- type: nauc_mrr_at_3_max
value: 50.048470016569354
- type: nauc_mrr_at_3_std
value: 49.28792737345567
- type: nauc_mrr_at_5_diff1
value: 40.13322604246314
- type: nauc_mrr_at_5_max
value: 50.66141477744915
- type: nauc_mrr_at_5_std
value: 49.80918618326533
- type: nauc_ndcg_at_1000_diff1
value: 36.607140549769674
- type: nauc_ndcg_at_1000_max
value: 59.200587723741116
- type: nauc_ndcg_at_1000_std
value: 58.43352243380002
- type: nauc_ndcg_at_100_diff1
value: 36.73336902084417
- type: nauc_ndcg_at_100_max
value: 58.519961959409684
- type: nauc_ndcg_at_100_std
value: 56.224789840949754
- type: nauc_ndcg_at_10_diff1
value: 36.68557551060416
- type: nauc_ndcg_at_10_max
value: 53.51807884301354
- type: nauc_ndcg_at_10_std
value: 47.00573027802542
- type: nauc_ndcg_at_1_diff1
value: 40.89200947569173
- type: nauc_ndcg_at_1_max
value: 43.97379094936904
- type: nauc_ndcg_at_1_std
value: 40.66402564181813
- type: nauc_ndcg_at_20_diff1
value: 37.59320916137791
- type: nauc_ndcg_at_20_max
value: 55.4760546361843
- type: nauc_ndcg_at_20_std
value: 48.90767410669467
- type: nauc_ndcg_at_3_diff1
value: 33.84089957019968
- type: nauc_ndcg_at_3_max
value: 47.47016859565707
- type: nauc_ndcg_at_3_std
value: 42.37468925179588
- type: nauc_ndcg_at_5_diff1
value: 36.10460149053392
- type: nauc_ndcg_at_5_max
value: 50.69500182543517
- type: nauc_ndcg_at_5_std
value: 44.92338706293522
- type: nauc_precision_at_1000_diff1
value: 3.1754413675556146
- type: nauc_precision_at_1000_max
value: 16.330732559063183
- type: nauc_precision_at_1000_std
value: 22.277465209117686
- type: nauc_precision_at_100_diff1
value: 17.836751231601824
- type: nauc_precision_at_100_max
value: 42.322558777461616
- type: nauc_precision_at_100_std
value: 54.04369812820183
- type: nauc_precision_at_10_diff1
value: 27.93529925212419
- type: nauc_precision_at_10_max
value: 50.58765423666032
- type: nauc_precision_at_10_std
value: 51.308076625252305
- type: nauc_precision_at_1_diff1
value: 43.60342597401739
- type: nauc_precision_at_1_max
value: 46.72967688790819
- type: nauc_precision_at_1_std
value: 46.08259257176318
- type: nauc_precision_at_20_diff1
value: 24.695126012289926
- type: nauc_precision_at_20_max
value: 48.97822472911428
- type: nauc_precision_at_20_std
value: 52.78431022058864
- type: nauc_precision_at_3_diff1
value: 30.002783640442022
- type: nauc_precision_at_3_max
value: 50.31584745650375
- type: nauc_precision_at_3_std
value: 47.50893636951425
- type: nauc_precision_at_5_diff1
value: 29.742591793684444
- type: nauc_precision_at_5_max
value: 50.085816529536146
- type: nauc_precision_at_5_std
value: 48.42214490630529
- type: nauc_recall_at_1000_diff1
value: 31.738216937067165
- type: nauc_recall_at_1000_max
value: 54.75637307089294
- type: nauc_recall_at_1000_std
value: 56.572334051110865
- type: nauc_recall_at_100_diff1
value: 31.16775594856223
- type: nauc_recall_at_100_max
value: 54.92671005034537
- type: nauc_recall_at_100_std
value: 54.546788497779715
- type: nauc_recall_at_10_diff1
value: 33.10538844811114
- type: nauc_recall_at_10_max
value: 50.619980861120716
- type: nauc_recall_at_10_std
value: 38.51495002919943
- type: nauc_recall_at_1_diff1
value: 43.38566026898597
- type: nauc_recall_at_1_max
value: 37.14628586578234
- type: nauc_recall_at_1_std
value: 26.2042447918162
- type: nauc_recall_at_20_diff1
value: 32.07750251540349
- type: nauc_recall_at_20_max
value: 50.293350866314626
- type: nauc_recall_at_20_std
value: 40.35085495599968
- type: nauc_recall_at_3_diff1
value: 31.048599549673312
- type: nauc_recall_at_3_max
value: 42.07116197279289
- type: nauc_recall_at_3_std
value: 29.31843910775867
- type: nauc_recall_at_5_diff1
value: 33.92167978742597
- type: nauc_recall_at_5_max
value: 47.34140185112234
- type: nauc_recall_at_5_std
value: 34.884310458067155
- type: ndcg_at_1
value: 27.125
- type: ndcg_at_10
value: 20.819
- type: ndcg_at_100
value: 23.602
- type: ndcg_at_1000
value: 28.147
- type: ndcg_at_20
value: 20.467
- type: ndcg_at_3
value: 23.854
- type: ndcg_at_5
value: 22.476
- type: precision_at_1
value: 34.25
- type: precision_at_10
value: 17.575
- type: precision_at_100
value: 5.668
- type: precision_at_1000
value: 1.149
- type: precision_at_20
value: 13.111999999999998
- type: precision_at_3
value: 27.083000000000002
- type: precision_at_5
value: 23.25
- type: recall_at_1
value: 3.628
- type: recall_at_10
value: 12.722
- type: recall_at_100
value: 28.481
- type: recall_at_1000
value: 44.385999999999996
- type: recall_at_20
value: 17.039
- type: recall_at_3
value: 7.600999999999999
- type: recall_at_5
value: 9.77
task:
type: Retrieval
- dataset:
config: default
name: MTEB DuRetrieval (default)
revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
split: test
type: C-MTEB/DuRetrieval
metrics:
- type: main_score
value: 68.562
- type: map_at_1
value: 20.598
- type: map_at_10
value: 58.291000000000004
- type: map_at_100
value: 61.214
- type: map_at_1000
value: 61.336999999999996
- type: map_at_20
value: 60.29600000000001
- type: map_at_3
value: 41.424
- type: map_at_5
value: 51.158
- type: mrr_at_1
value: 76.4
- type: mrr_at_10
value: 82.07696428571425
- type: mrr_at_100
value: 82.23728543548712
- type: mrr_at_1000
value: 82.2484704491435
- type: mrr_at_20
value: 82.17939404012738
- type: mrr_at_3
value: 81.39999999999996
- type: mrr_at_5
value: 81.77749999999999
- type: nauc_map_at_1000_diff1
value: 12.062132936717521
- type: nauc_map_at_1000_max
value: 51.382165085093334
- type: nauc_map_at_1000_std
value: 36.58864306046064
- type: nauc_map_at_100_diff1
value: 12.051949398444183
- type: nauc_map_at_100_max
value: 51.34908135366227
- type: nauc_map_at_100_std
value: 36.52210989937193
- type: nauc_map_at_10_diff1
value: 12.777280190641028
- type: nauc_map_at_10_max
value: 49.32596164262907
- type: nauc_map_at_10_std
value: 31.686981061919155
- type: nauc_map_at_1_diff1
value: 37.35139732613343
- type: nauc_map_at_1_max
value: 15.093384481462307
- type: nauc_map_at_1_std
value: -8.854873266650177
- type: nauc_map_at_20_diff1
value: 12.021476990878154
- type: nauc_map_at_20_max
value: 51.03680970054935
- type: nauc_map_at_20_std
value: 35.39711804826195
- type: nauc_map_at_3_diff1
value: 21.411553096160677
- type: nauc_map_at_3_max
value: 33.17045029624202
- type: nauc_map_at_3_std
value: 8.535144908242547
- type: nauc_map_at_5_diff1
value: 15.708436109180901
- type: nauc_map_at_5_max
value: 42.409321391070755
- type: nauc_map_at_5_std
value: 20.4389846931215
- type: nauc_mrr_at_1000_diff1
value: 38.428414816579085
- type: nauc_mrr_at_1000_max
value: 63.60673073805357
- type: nauc_mrr_at_1000_std
value: 40.71970714381069
- type: nauc_mrr_at_100_diff1
value: 38.42980859765321
- type: nauc_mrr_at_100_max
value: 63.619004356960005
- type: nauc_mrr_at_100_std
value: 40.72448587110982
- type: nauc_mrr_at_10_diff1
value: 38.47826112008598
- type: nauc_mrr_at_10_max
value: 63.67807682471137
- type: nauc_mrr_at_10_std
value: 40.72466524474895
- type: nauc_mrr_at_1_diff1
value: 39.283302491602534
- type: nauc_mrr_at_1_max
value: 59.024604685019625
- type: nauc_mrr_at_1_std
value: 34.94700630574531
- type: nauc_mrr_at_20_diff1
value: 38.426716863570896
- type: nauc_mrr_at_20_max
value: 63.672103746346245
- type: nauc_mrr_at_20_std
value: 40.787049239349656
- type: nauc_mrr_at_3_diff1
value: 37.89957558233111
- type: nauc_mrr_at_3_max
value: 63.69612725658364
- type: nauc_mrr_at_3_std
value: 40.65582656561661
- type: nauc_mrr_at_5_diff1
value: 37.979706587392855
- type: nauc_mrr_at_5_max
value: 63.596719178013686
- type: nauc_mrr_at_5_std
value: 40.679370708153975
- type: nauc_ndcg_at_1000_diff1
value: 15.943236492583047
- type: nauc_ndcg_at_1000_max
value: 56.687522809123735
- type: nauc_ndcg_at_1000_std
value: 43.7481773406075
- type: nauc_ndcg_at_100_diff1
value: 15.570990846108915
- type: nauc_ndcg_at_100_max
value: 56.39526331713808
- type: nauc_ndcg_at_100_std
value: 43.39751374918785
- type: nauc_ndcg_at_10_diff1
value: 15.48802688248517
- type: nauc_ndcg_at_10_max
value: 54.59534251030818
- type: nauc_ndcg_at_10_std
value: 38.824175992946984
- type: nauc_ndcg_at_1_diff1
value: 39.283302491602534
- type: nauc_ndcg_at_1_max
value: 59.024604685019625
- type: nauc_ndcg_at_1_std
value: 34.94700630574531
- type: nauc_ndcg_at_20_diff1
value: 15.068158050742674
- type: nauc_ndcg_at_20_max
value: 55.981346078084215
- type: nauc_ndcg_at_20_std
value: 41.43625902852297
- type: nauc_ndcg_at_3_diff1
value: 13.918883919943884
- type: nauc_ndcg_at_3_max
value: 54.017797306767875
- type: nauc_ndcg_at_3_std
value: 37.783006567845504
- type: nauc_ndcg_at_5_diff1
value: 13.828905669873098
- type: nauc_ndcg_at_5_max
value: 52.855256667797576
- type: nauc_ndcg_at_5_std
value: 36.1342649347302
- type: nauc_precision_at_1000_diff1
value: -13.4008203486104
- type: nauc_precision_at_1000_max
value: 20.34126695599174
- type: nauc_precision_at_1000_std
value: 39.759981698846275
- type: nauc_precision_at_100_diff1
value: -12.477594463684301
- type: nauc_precision_at_100_max
value: 26.1710872492505
- type: nauc_precision_at_100_std
value: 44.765114967812465
- type: nauc_precision_at_10_diff1
value: -10.139326403217545
- type: nauc_precision_at_10_max
value: 37.88081495944228
- type: nauc_precision_at_10_std
value: 46.70161705980195
- type: nauc_precision_at_1_diff1
value: 39.283302491602534
- type: nauc_precision_at_1_max
value: 59.024604685019625
- type: nauc_precision_at_1_std
value: 34.94700630574531
- type: nauc_precision_at_20_diff1
value: -11.639337184169326
- type: nauc_precision_at_20_max
value: 33.58893256534197
- type: nauc_precision_at_20_std
value: 47.50774316764828
- type: nauc_precision_at_3_diff1
value: -2.4077437192506648
- type: nauc_precision_at_3_max
value: 49.819809006594255
- type: nauc_precision_at_3_std
value: 43.314868908413565
- type: nauc_precision_at_5_diff1
value: -8.087192297095333
- type: nauc_precision_at_5_max
value: 44.25420534145421
- type: nauc_precision_at_5_std
value: 44.52972613962522
- type: nauc_recall_at_1000_diff1
value: 7.280604579210675
- type: nauc_recall_at_1000_max
value: 48.462169604520135
- type: nauc_recall_at_1000_std
value: 54.611400371209875
- type: nauc_recall_at_100_diff1
value: 7.855570930555611
- type: nauc_recall_at_100_max
value: 49.362395797047725
- type: nauc_recall_at_100_std
value: 48.21464586001012
- type: nauc_recall_at_10_diff1
value: 11.04991814644292
- type: nauc_recall_at_10_max
value: 47.31513207971589
- type: nauc_recall_at_10_std
value: 33.36665734051043
- type: nauc_recall_at_1_diff1
value: 37.35139732613343
- type: nauc_recall_at_1_max
value: 15.093384481462307
- type: nauc_recall_at_1_std
value: -8.854873266650177
- type: nauc_recall_at_20_diff1
value: 8.043639724282265
- type: nauc_recall_at_20_max
value: 50.68059498542623
- type: nauc_recall_at_20_std
value: 42.43323496160291
- type: nauc_recall_at_3_diff1
value: 18.165808605381066
- type: nauc_recall_at_3_max
value: 29.433129898537082
- type: nauc_recall_at_3_std
value: 6.665891212065862
- type: nauc_recall_at_5_diff1
value: 12.40708163880872
- type: nauc_recall_at_5_max
value: 38.422720938138085
- type: nauc_recall_at_5_std
value: 18.382267523266844
- type: ndcg_at_1
value: 76.4
- type: ndcg_at_10
value: 68.562
- type: ndcg_at_100
value: 73.12100000000001
- type: ndcg_at_1000
value: 74.598
- type: ndcg_at_20
value: 70.60600000000001
- type: ndcg_at_3
value: 69.203
- type: ndcg_at_5
value: 66.8
- type: precision_at_1
value: 76.4
- type: precision_at_10
value: 32.714999999999996
- type: precision_at_100
value: 4.089
- type: precision_at_1000
value: 0.445
- type: precision_at_20
value: 18.203
- type: precision_at_3
value: 61.833000000000006
- type: precision_at_5
value: 50.86000000000001
- type: recall_at_1
value: 20.598
- type: recall_at_10
value: 68.785
- type: recall_at_100
value: 83.14099999999999
- type: recall_at_1000
value: 90.726
- type: recall_at_20
value: 74.811
- type: recall_at_3
value: 44.346000000000004
- type: recall_at_5
value: 56.79
task:
type: Retrieval
- dataset:
config: default
name: MTEB EcomRetrieval (default)
revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
split: test
type: C-MTEB/EcomRetrieval
metrics:
- type: main_score
value: 51.99399999999999
- type: map_at_1
value: 39
- type: map_at_10
value: 47.498000000000005
- type: map_at_100
value: 48.271
- type: map_at_1000
value: 48.306
- type: map_at_20
value: 47.965
- type: map_at_3
value: 45.25
- type: map_at_5
value: 46.465
- type: mrr_at_1
value: 39
- type: mrr_at_10
value: 47.49777777777774
- type: mrr_at_100
value: 48.27105812675479
- type: mrr_at_1000
value: 48.3064507593175
- type: mrr_at_20
value: 47.965205475638854
- type: mrr_at_3
value: 45.24999999999997
- type: mrr_at_5
value: 46.464999999999954
- type: nauc_map_at_1000_diff1
value: 64.14211914604797
- type: nauc_map_at_1000_max
value: 43.48928410278362
- type: nauc_map_at_1000_std
value: -0.16916880177790458
- type: nauc_map_at_100_diff1
value: 64.1326333446995
- type: nauc_map_at_100_max
value: 43.50413257508422
- type: nauc_map_at_100_std
value: -0.16017811389068248
- type: nauc_map_at_10_diff1
value: 64.19919182450525
- type: nauc_map_at_10_max
value: 43.42535989882292
- type: nauc_map_at_10_std
value: -0.5926134329060908
- type: nauc_map_at_1_diff1
value: 67.13476388188273
- type: nauc_map_at_1_max
value: 38.74949879063031
- type: nauc_map_at_1_std
value: -2.1603093916935094
- type: nauc_map_at_20_diff1
value: 64.10452313276582
- type: nauc_map_at_20_max
value: 43.46062026023969
- type: nauc_map_at_20_std
value: -0.328594445444764
- type: nauc_map_at_3_diff1
value: 64.62046082110288
- type: nauc_map_at_3_max
value: 42.202412793697306
- type: nauc_map_at_3_std
value: -0.8994670765839452
- type: nauc_map_at_5_diff1
value: 64.25068056127812
- type: nauc_map_at_5_max
value: 42.694992011748326
- type: nauc_map_at_5_std
value: -0.9028996258442468
- type: nauc_mrr_at_1000_diff1
value: 64.14211914604797
- type: nauc_mrr_at_1000_max
value: 43.48928410278362
- type: nauc_mrr_at_1000_std
value: -0.16916880177790458
- type: nauc_mrr_at_100_diff1
value: 64.1326333446995
- type: nauc_mrr_at_100_max
value: 43.50413257508422
- type: nauc_mrr_at_100_std
value: -0.16017811389068248
- type: nauc_mrr_at_10_diff1
value: 64.19919182450525
- type: nauc_mrr_at_10_max
value: 43.42535989882292
- type: nauc_mrr_at_10_std
value: -0.5926134329060908
- type: nauc_mrr_at_1_diff1
value: 67.13476388188273
- type: nauc_mrr_at_1_max
value: 38.74949879063031
- type: nauc_mrr_at_1_std
value: -2.1603093916935094
- type: nauc_mrr_at_20_diff1
value: 64.10452313276582
- type: nauc_mrr_at_20_max
value: 43.46062026023969
- type: nauc_mrr_at_20_std
value: -0.328594445444764
- type: nauc_mrr_at_3_diff1
value: 64.62046082110288
- type: nauc_mrr_at_3_max
value: 42.202412793697306
- type: nauc_mrr_at_3_std
value: -0.8994670765839452
- type: nauc_mrr_at_5_diff1
value: 64.25068056127812
- type: nauc_mrr_at_5_max
value: 42.694992011748326
- type: nauc_mrr_at_5_std
value: -0.9028996258442468
- type: nauc_ndcg_at_1000_diff1
value: 63.21768899139219
- type: nauc_ndcg_at_1000_max
value: 46.19506099094367
- type: nauc_ndcg_at_1000_std
value: 2.209865028223472
- type: nauc_ndcg_at_100_diff1
value: 62.98633379848151
- type: nauc_ndcg_at_100_max
value: 46.63979239002217
- type: nauc_ndcg_at_100_std
value: 2.695628752678766
- type: nauc_ndcg_at_10_diff1
value: 62.99364854584548
- type: nauc_ndcg_at_10_max
value: 45.922535262934275
- type: nauc_ndcg_at_10_std
value: 0.23158851918729126
- type: nauc_ndcg_at_1_diff1
value: 67.13476388188273
- type: nauc_ndcg_at_1_max
value: 38.74949879063031
- type: nauc_ndcg_at_1_std
value: -2.1603093916935094
- type: nauc_ndcg_at_20_diff1
value: 62.7430226696847
- type: nauc_ndcg_at_20_max
value: 46.24633935920644
- type: nauc_ndcg_at_20_std
value: 1.2933574402770873
- type: nauc_ndcg_at_3_diff1
value: 63.779909868594075
- type: nauc_ndcg_at_3_max
value: 43.195971704826555
- type: nauc_ndcg_at_3_std
value: -0.4697879751828786
- type: nauc_ndcg_at_5_diff1
value: 63.077014069645465
- type: nauc_ndcg_at_5_max
value: 44.05823309172281
- type: nauc_ndcg_at_5_std
value: -0.516902229420011
- type: nauc_precision_at_1000_diff1
value: 57.28354404824984
- type: nauc_precision_at_1000_max
value: 81.1870694223634
- type: nauc_precision_at_1000_std
value: 39.5740278093216
- type: nauc_precision_at_100_diff1
value: 56.348657337866
- type: nauc_precision_at_100_max
value: 72.27778216986857
- type: nauc_precision_at_100_std
value: 31.08776714891819
- type: nauc_precision_at_10_diff1
value: 58.654066064682354
- type: nauc_precision_at_10_max
value: 55.66851802697398
- type: nauc_precision_at_10_std
value: 3.4120787487541695
- type: nauc_precision_at_1_diff1
value: 67.13476388188273
- type: nauc_precision_at_1_max
value: 38.74949879063031
- type: nauc_precision_at_1_std
value: -2.1603093916935094
- type: nauc_precision_at_20_diff1
value: 56.95993841868541
- type: nauc_precision_at_20_max
value: 59.520800557088606
- type: nauc_precision_at_20_std
value: 9.678027334643966
- type: nauc_precision_at_3_diff1
value: 61.25203222883966
- type: nauc_precision_at_3_max
value: 46.132939137245586
- type: nauc_precision_at_3_std
value: 0.8250263034877992
- type: nauc_precision_at_5_diff1
value: 59.27696427138276
- type: nauc_precision_at_5_max
value: 48.3729142498595
- type: nauc_precision_at_5_std
value: 0.6543072062803423
- type: nauc_recall_at_1000_diff1
value: 57.283544048249965
- type: nauc_recall_at_1000_max
value: 81.18706942236348
- type: nauc_recall_at_1000_std
value: 39.574027809322
- type: nauc_recall_at_100_diff1
value: 56.348657337865895
- type: nauc_recall_at_100_max
value: 72.27778216986839
- type: nauc_recall_at_100_std
value: 31.08776714891815
- type: nauc_recall_at_10_diff1
value: 58.65406606468243
- type: nauc_recall_at_10_max
value: 55.6685180269741
- type: nauc_recall_at_10_std
value: 3.4120787487541744
- type: nauc_recall_at_1_diff1
value: 67.13476388188273
- type: nauc_recall_at_1_max
value: 38.74949879063031
- type: nauc_recall_at_1_std
value: -2.1603093916935094
- type: nauc_recall_at_20_diff1
value: 56.95993841868553
- type: nauc_recall_at_20_max
value: 59.520800557088705
- type: nauc_recall_at_20_std
value: 9.67802733464404
- type: nauc_recall_at_3_diff1
value: 61.25203222883967
- type: nauc_recall_at_3_max
value: 46.132939137245586
- type: nauc_recall_at_3_std
value: 0.8250263034877896
- type: nauc_recall_at_5_diff1
value: 59.27696427138285
- type: nauc_recall_at_5_max
value: 48.372914249859576
- type: nauc_recall_at_5_std
value: 0.6543072062804443
- type: ndcg_at_1
value: 39
- type: ndcg_at_10
value: 51.99399999999999
- type: ndcg_at_100
value: 55.95099999999999
- type: ndcg_at_1000
value: 56.885
- type: ndcg_at_20
value: 53.74
- type: ndcg_at_3
value: 47.315000000000005
- type: ndcg_at_5
value: 49.480000000000004
- type: precision_at_1
value: 39
- type: precision_at_10
value: 6.63
- type: precision_at_100
value: 0.852
- type: precision_at_1000
value: 0.093
- type: precision_at_20
value: 3.665
- type: precision_at_3
value: 17.767
- type: precision_at_5
value: 11.700000000000001
- type: recall_at_1
value: 39
- type: recall_at_10
value: 66.3
- type: recall_at_100
value: 85.2
- type: recall_at_1000
value: 92.60000000000001
- type: recall_at_20
value: 73.3
- type: recall_at_3
value: 53.300000000000004
- type: recall_at_5
value: 58.5
task:
type: Retrieval
- dataset:
config: default
name: MTEB FEVER (default)
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
split: test
type: mteb/fever
metrics:
- type: main_score
value: 86.167
- type: map_at_1
value: 74.33800000000001
- type: map_at_10
value: 82.498
- type: map_at_100
value: 82.711
- type: map_at_1000
value: 82.722
- type: map_at_20
value: 82.631
- type: map_at_3
value: 81.366
- type: map_at_5
value: 82.087
- type: mrr_at_1
value: 79.8079807980798
- type: mrr_at_10
value: 86.81534820148663
- type: mrr_at_100
value: 86.90335837526817
- type: mrr_at_1000
value: 86.90570029953601
- type: mrr_at_20
value: 86.87497576561636
- type: mrr_at_3
value: 86.03110311031085
- type: mrr_at_5
value: 86.56515651565132
- type: nauc_map_at_1000_diff1
value: 63.80412003468792
- type: nauc_map_at_1000_max
value: 30.443040219063782
- type: nauc_map_at_1000_std
value: -0.7661699065712747
- type: nauc_map_at_100_diff1
value: 63.793185543664144
- type: nauc_map_at_100_max
value: 30.449106167217906
- type: nauc_map_at_100_std
value: -0.751513142093214
- type: nauc_map_at_10_diff1
value: 63.6468809138792
- type: nauc_map_at_10_max
value: 30.468882268790736
- type: nauc_map_at_10_std
value: -0.8770760767073671
- type: nauc_map_at_1_diff1
value: 65.59476914772542
- type: nauc_map_at_1_max
value: 24.8326129481821
- type: nauc_map_at_1_std
value: -6.149274398563455
- type: nauc_map_at_20_diff1
value: 63.72389441483504
- type: nauc_map_at_20_max
value: 30.457609974874234
- type: nauc_map_at_20_std
value: -0.808031265800565
- type: nauc_map_at_3_diff1
value: 63.41453934510057
- type: nauc_map_at_3_max
value: 30.12460449412893
- type: nauc_map_at_3_std
value: -1.6479523970241587
- type: nauc_map_at_5_diff1
value: 63.72460033731496
- type: nauc_map_at_5_max
value: 30.641849557038064
- type: nauc_map_at_5_std
value: -1.001988906727461
- type: nauc_mrr_at_1000_diff1
value: 76.36838740180266
- type: nauc_mrr_at_1000_max
value: 35.08826457750934
- type: nauc_mrr_at_1000_std
value: -0.6777459508169168
- type: nauc_mrr_at_100_diff1
value: 76.36608691519517
- type: nauc_mrr_at_100_max
value: 35.09638914785759
- type: nauc_mrr_at_100_std
value: -0.6634803913379186
- type: nauc_mrr_at_10_diff1
value: 76.3948904658589
- type: nauc_mrr_at_10_max
value: 35.33356677586711
- type: nauc_mrr_at_10_std
value: -0.45659136145068113
- type: nauc_mrr_at_1_diff1
value: 76.68686096982084
- type: nauc_mrr_at_1_max
value: 29.761571677851446
- type: nauc_mrr_at_1_std
value: -5.907589944585199
- type: nauc_mrr_at_20_diff1
value: 76.37370624306932
- type: nauc_mrr_at_20_max
value: 35.19943783422336
- type: nauc_mrr_at_20_std
value: -0.5618030993571618
- type: nauc_mrr_at_3_diff1
value: 76.11716503186365
- type: nauc_mrr_at_3_max
value: 35.271244192469
- type: nauc_mrr_at_3_std
value: -0.8712763707650889
- type: nauc_mrr_at_5_diff1
value: 76.48015586019038
- type: nauc_mrr_at_5_max
value: 35.66063047390566
- type: nauc_mrr_at_5_std
value: -0.2899088302722794
- type: nauc_ndcg_at_1000_diff1
value: 65.09177399227487
- type: nauc_ndcg_at_1000_max
value: 32.44696464141599
- type: nauc_ndcg_at_1000_std
value: 1.4136160879212092
- type: nauc_ndcg_at_100_diff1
value: 64.74921044658977
- type: nauc_ndcg_at_100_max
value: 32.566752866071425
- type: nauc_ndcg_at_100_std
value: 1.855958522657113
- type: nauc_ndcg_at_10_diff1
value: 64.28645210451032
- type: nauc_ndcg_at_10_max
value: 33.14957686106596
- type: nauc_ndcg_at_10_std
value: 2.0113075275468355
- type: nauc_ndcg_at_1_diff1
value: 76.68686096982084
- type: nauc_ndcg_at_1_max
value: 29.761571677851446
- type: nauc_ndcg_at_1_std
value: -5.907589944585199
- type: nauc_ndcg_at_20_diff1
value: 64.39446221768821
- type: nauc_ndcg_at_20_max
value: 32.93545770746945
- type: nauc_ndcg_at_20_std
value: 1.9924181843944653
- type: nauc_ndcg_at_3_diff1
value: 64.79283621071
- type: nauc_ndcg_at_3_max
value: 32.7746091928708
- type: nauc_ndcg_at_3_std
value: 0.5288567628664994
- type: nauc_ndcg_at_5_diff1
value: 64.83506189605514
- type: nauc_ndcg_at_5_max
value: 33.67363285662579
- type: nauc_ndcg_at_5_std
value: 1.8119931952532888
- type: nauc_precision_at_1000_diff1
value: -15.714725289554032
- type: nauc_precision_at_1000_max
value: -4.37623120107013
- type: nauc_precision_at_1000_std
value: 9.293828089569525
- type: nauc_precision_at_100_diff1
value: -13.216686619671536
- type: nauc_precision_at_100_max
value: 0.16415424652433566
- type: nauc_precision_at_100_std
value: 14.673489903735938
- type: nauc_precision_at_10_diff1
value: 1.62676209843492
- type: nauc_precision_at_10_max
value: 15.005360115331227
- type: nauc_precision_at_10_std
value: 17.90978542772984
- type: nauc_precision_at_1_diff1
value: 76.68686096982084
- type: nauc_precision_at_1_max
value: 29.761571677851446
- type: nauc_precision_at_1_std
value: -5.907589944585199
- type: nauc_precision_at_20_diff1
value: -5.553830318468026
- type: nauc_precision_at_20_max
value: 8.735585741431837
- type: nauc_precision_at_20_std
value: 16.964680678765866
- type: nauc_precision_at_3_diff1
value: 34.33896583653477
- type: nauc_precision_at_3_max
value: 31.545012727670148
- type: nauc_precision_at_3_std
value: 12.5377712701041
- type: nauc_precision_at_5_diff1
value: 19.490544912279802
- type: nauc_precision_at_5_max
value: 26.830224222236104
- type: nauc_precision_at_5_std
value: 17.923062797715478
- type: nauc_recall_at_1000_diff1
value: 22.97875537876177
- type: nauc_recall_at_1000_max
value: 29.35608496530822
- type: nauc_recall_at_1000_std
value: 23.12406199891762
- type: nauc_recall_at_100_diff1
value: 29.298613044090967
- type: nauc_recall_at_100_max
value: 31.992797356607166
- type: nauc_recall_at_100_std
value: 25.076009885245366
- type: nauc_recall_at_10_diff1
value: 41.243537896155836
- type: nauc_recall_at_10_max
value: 37.63206937207168
- type: nauc_recall_at_10_std
value: 18.18023942405805
- type: nauc_recall_at_1_diff1
value: 65.59476914772542
- type: nauc_recall_at_1_max
value: 24.8326129481821
- type: nauc_recall_at_1_std
value: -6.149274398563455
- type: nauc_recall_at_20_diff1
value: 36.94224436204051
- type: nauc_recall_at_20_max
value: 36.4121436479055
- type: nauc_recall_at_20_std
value: 20.56212393263084
- type: nauc_recall_at_3_diff1
value: 52.03839901501732
- type: nauc_recall_at_3_max
value: 35.5899002298603
- type: nauc_recall_at_3_std
value: 7.407128983355843
- type: nauc_recall_at_5_diff1
value: 49.02745857931576
- type: nauc_recall_at_5_max
value: 39.06314797493658
- type: nauc_recall_at_5_std
value: 13.613515240830148
- type: ndcg_at_1
value: 79.808
- type: ndcg_at_10
value: 86.167
- type: ndcg_at_100
value: 86.94800000000001
- type: ndcg_at_1000
value: 87.164
- type: ndcg_at_20
value: 86.529
- type: ndcg_at_3
value: 84.40299999999999
- type: ndcg_at_5
value: 85.40299999999999
- type: precision_at_1
value: 79.808
- type: precision_at_10
value: 10.345
- type: precision_at_100
value: 1.095
- type: precision_at_1000
value: 0.11299999999999999
- type: precision_at_20
value: 5.284
- type: precision_at_3
value: 32.178000000000004
- type: precision_at_5
value: 20.003
- type: recall_at_1
value: 74.33800000000001
- type: recall_at_10
value: 93.125
- type: recall_at_100
value: 96.26599999999999
- type: recall_at_1000
value: 97.669
- type: recall_at_20
value: 94.36
- type: recall_at_3
value: 88.333
- type: recall_at_5
value: 90.892
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018 (default)
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
split: test
type: mteb/fiqa
metrics:
- type: main_score
value: 25.805
- type: map_at_1
value: 12.899
- type: map_at_10
value: 19.764
- type: map_at_100
value: 20.96
- type: map_at_1000
value: 21.124000000000002
- type: map_at_20
value: 20.407
- type: map_at_3
value: 17.18
- type: map_at_5
value: 18.481
- type: mrr_at_1
value: 24.382716049382715
- type: mrr_at_10
value: 31.675117577895325
- type: mrr_at_100
value: 32.44952245761694
- type: mrr_at_1000
value: 32.512907986779325
- type: mrr_at_20
value: 32.112282911664416
- type: mrr_at_3
value: 29.37242798353909
- type: mrr_at_5
value: 30.483539094650187
- type: nauc_map_at_1000_diff1
value: 40.54385359909451
- type: nauc_map_at_1000_max
value: 38.74918403533164
- type: nauc_map_at_1000_std
value: 15.43983176915499
- type: nauc_map_at_100_diff1
value: 40.473604319109626
- type: nauc_map_at_100_max
value: 38.685313962942416
- type: nauc_map_at_100_std
value: 15.35390348208144
- type: nauc_map_at_10_diff1
value: 40.57894981383912
- type: nauc_map_at_10_max
value: 38.13086669076117
- type: nauc_map_at_10_std
value: 14.17584341830685
- type: nauc_map_at_1_diff1
value: 47.369181938582514
- type: nauc_map_at_1_max
value: 36.51527795473331
- type: nauc_map_at_1_std
value: 13.579054537840229
- type: nauc_map_at_20_diff1
value: 40.40176108135633
- type: nauc_map_at_20_max
value: 38.33885949371027
- type: nauc_map_at_20_std
value: 14.775922981863355
- type: nauc_map_at_3_diff1
value: 42.34450676309069
- type: nauc_map_at_3_max
value: 36.81828677228458
- type: nauc_map_at_3_std
value: 12.53054692016046
- type: nauc_map_at_5_diff1
value: 41.4195127461581
- type: nauc_map_at_5_max
value: 37.43951911767563
- type: nauc_map_at_5_std
value: 12.988499451959793
- type: nauc_mrr_at_1000_diff1
value: 42.32368879629551
- type: nauc_mrr_at_1000_max
value: 41.49392233754857
- type: nauc_mrr_at_1000_std
value: 18.59248217498413
- type: nauc_mrr_at_100_diff1
value: 42.26325978558565
- type: nauc_mrr_at_100_max
value: 41.4827305847626
- type: nauc_mrr_at_100_std
value: 18.593747709016778
- type: nauc_mrr_at_10_diff1
value: 42.1690403669623
- type: nauc_mrr_at_10_max
value: 41.407523945768546
- type: nauc_mrr_at_10_std
value: 18.359603563477144
- type: nauc_mrr_at_1_diff1
value: 48.53953775909429
- type: nauc_mrr_at_1_max
value: 44.51007496638179
- type: nauc_mrr_at_1_std
value: 18.27945359299317
- type: nauc_mrr_at_20_diff1
value: 42.246192660407495
- type: nauc_mrr_at_20_max
value: 41.492150403414755
- type: nauc_mrr_at_20_std
value: 18.557913585288404
- type: nauc_mrr_at_3_diff1
value: 43.56499280912534
- type: nauc_mrr_at_3_max
value: 40.63393511031206
- type: nauc_mrr_at_3_std
value: 16.54470864684114
- type: nauc_mrr_at_5_diff1
value: 42.96188069849647
- type: nauc_mrr_at_5_max
value: 41.29469780945155
- type: nauc_mrr_at_5_std
value: 17.589214122351695
- type: nauc_ndcg_at_1000_diff1
value: 38.997425615002776
- type: nauc_ndcg_at_1000_max
value: 41.18105491785995
- type: nauc_ndcg_at_1000_std
value: 21.07194878703698
- type: nauc_ndcg_at_100_diff1
value: 38.05358129703645
- type: nauc_ndcg_at_100_max
value: 40.4891814552976
- type: nauc_ndcg_at_100_std
value: 20.18146477851188
- type: nauc_ndcg_at_10_diff1
value: 38.0786204666326
- type: nauc_ndcg_at_10_max
value: 38.73202623740625
- type: nauc_ndcg_at_10_std
value: 16.3553298180595
- type: nauc_ndcg_at_1_diff1
value: 48.53953775909429
- type: nauc_ndcg_at_1_max
value: 44.51007496638179
- type: nauc_ndcg_at_1_std
value: 18.27945359299317
- type: nauc_ndcg_at_20_diff1
value: 37.781506163040454
- type: nauc_ndcg_at_20_max
value: 39.185888526787885
- type: nauc_ndcg_at_20_std
value: 17.877395044750358
- type: nauc_ndcg_at_3_diff1
value: 40.01423266022924
- type: nauc_ndcg_at_3_max
value: 37.64027658055322
- type: nauc_ndcg_at_3_std
value: 12.568977177168858
- type: nauc_ndcg_at_5_diff1
value: 39.50800617745676
- type: nauc_ndcg_at_5_max
value: 38.1027458795071
- type: nauc_ndcg_at_5_std
value: 14.078522330619242
- type: nauc_precision_at_1000_diff1
value: 5.960260178366493
- type: nauc_precision_at_1000_max
value: 20.2434346099095
- type: nauc_precision_at_1000_std
value: 14.879347523960806
- type: nauc_precision_at_100_diff1
value: 14.365600944912302
- type: nauc_precision_at_100_max
value: 29.70429196562339
- type: nauc_precision_at_100_std
value: 20.676479232937577
- type: nauc_precision_at_10_diff1
value: 23.11599184736198
- type: nauc_precision_at_10_max
value: 37.85650058533224
- type: nauc_precision_at_10_std
value: 19.717492234368024
- type: nauc_precision_at_1_diff1
value: 48.53953775909429
- type: nauc_precision_at_1_max
value: 44.51007496638179
- type: nauc_precision_at_1_std
value: 18.27945359299317
- type: nauc_precision_at_20_diff1
value: 18.940880127043123
- type: nauc_precision_at_20_max
value: 33.526263365412426
- type: nauc_precision_at_20_std
value: 19.159930760713596
- type: nauc_precision_at_3_diff1
value: 33.399845354841695
- type: nauc_precision_at_3_max
value: 40.104701275367475
- type: nauc_precision_at_3_std
value: 14.391437252550677
- type: nauc_precision_at_5_diff1
value: 30.242747760482825
- type: nauc_precision_at_5_max
value: 39.45089096851181
- type: nauc_precision_at_5_std
value: 15.471086249839974
- type: nauc_recall_at_1000_diff1
value: 26.932411631320768
- type: nauc_recall_at_1000_max
value: 36.9739917322874
- type: nauc_recall_at_1000_std
value: 34.75368398811744
- type: nauc_recall_at_100_diff1
value: 23.269700249461017
- type: nauc_recall_at_100_max
value: 34.30265666696766
- type: nauc_recall_at_100_std
value: 26.224754528985255
- type: nauc_recall_at_10_diff1
value: 25.61336594823433
- type: nauc_recall_at_10_max
value: 31.36165445449294
- type: nauc_recall_at_10_std
value: 14.716681480629642
- type: nauc_recall_at_1_diff1
value: 47.369181938582514
- type: nauc_recall_at_1_max
value: 36.51527795473331
- type: nauc_recall_at_1_std
value: 13.579054537840229
- type: nauc_recall_at_20_diff1
value: 24.208738126912145
- type: nauc_recall_at_20_max
value: 31.605126284041297
- type: nauc_recall_at_20_std
value: 18.58561210868371
- type: nauc_recall_at_3_diff1
value: 33.978557510518016
- type: nauc_recall_at_3_max
value: 30.416526848179316
- type: nauc_recall_at_3_std
value: 9.024784504488089
- type: nauc_recall_at_5_diff1
value: 31.45101894206735
- type: nauc_recall_at_5_max
value: 32.10370802263191
- type: nauc_recall_at_5_std
value: 11.398936864675383
- type: ndcg_at_1
value: 24.383
- type: ndcg_at_10
value: 25.805
- type: ndcg_at_100
value: 30.994
- type: ndcg_at_1000
value: 34.304
- type: ndcg_at_20
value: 27.645999999999997
- type: ndcg_at_3
value: 22.516
- type: ndcg_at_5
value: 23.506
- type: precision_at_1
value: 24.383
- type: precision_at_10
value: 7.037
- type: precision_at_100
value: 1.242
- type: precision_at_1000
value: 0.182
- type: precision_at_20
value: 4.329000000000001
- type: precision_at_3
value: 14.249
- type: precision_at_5
value: 10.648
- type: recall_at_1
value: 12.899
- type: recall_at_10
value: 31.938
- type: recall_at_100
value: 51.81
- type: recall_at_1000
value: 72.117
- type: recall_at_20
value: 37.483
- type: recall_at_3
value: 21.256
- type: recall_at_5
value: 25.259999999999998
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA (default)
revision: ab518f4d6fcca38d87c25209f94beba119d02014
split: test
type: mteb/hotpotqa
metrics:
- type: main_score
value: 67.472
- type: map_at_1
value: 39.426
- type: map_at_10
value: 58.462
- type: map_at_100
value: 59.384
- type: map_at_1000
value: 59.441
- type: map_at_20
value: 59.019
- type: map_at_3
value: 55.157999999999994
- type: map_at_5
value: 57.255
- type: mrr_at_1
value: 78.8521269412559
- type: mrr_at_10
value: 84.35582993044147
- type: mrr_at_100
value: 84.55263951304046
- type: mrr_at_1000
value: 84.55918448205495
- type: mrr_at_20
value: 84.49086444708152
- type: mrr_at_3
value: 83.39185235201421
- type: mrr_at_5
value: 84.04478955660558
- type: nauc_map_at_1000_diff1
value: 26.341340343769517
- type: nauc_map_at_1000_max
value: 28.527861182487207
- type: nauc_map_at_1000_std
value: 8.982598197502044
- type: nauc_map_at_100_diff1
value: 26.324274963576936
- type: nauc_map_at_100_max
value: 28.519595659496822
- type: nauc_map_at_100_std
value: 8.969958522705147
- type: nauc_map_at_10_diff1
value: 26.447737791862053
- type: nauc_map_at_10_max
value: 28.440489271707914
- type: nauc_map_at_10_std
value: 8.36496899684938
- type: nauc_map_at_1_diff1
value: 78.50105177648346
- type: nauc_map_at_1_max
value: 47.248470892515684
- type: nauc_map_at_1_std
value: -0.11060813067222093
- type: nauc_map_at_20_diff1
value: 26.31340728508947
- type: nauc_map_at_20_max
value: 28.476780271151476
- type: nauc_map_at_20_std
value: 8.708376028591845
- type: nauc_map_at_3_diff1
value: 27.623478336959227
- type: nauc_map_at_3_max
value: 28.25283797154275
- type: nauc_map_at_3_std
value: 6.717349104465825
- type: nauc_map_at_5_diff1
value: 26.75223688471738
- type: nauc_map_at_5_max
value: 28.480228558660865
- type: nauc_map_at_5_std
value: 7.777090164882629
- type: nauc_mrr_at_1000_diff1
value: 77.82566946312463
- type: nauc_mrr_at_1000_max
value: 50.978748622341776
- type: nauc_mrr_at_1000_std
value: 3.9660794347763457
- type: nauc_mrr_at_100_diff1
value: 77.82636195776996
- type: nauc_mrr_at_100_max
value: 50.989582370683074
- type: nauc_mrr_at_100_std
value: 3.9837834579455507
- type: nauc_mrr_at_10_diff1
value: 77.80859827606214
- type: nauc_mrr_at_10_max
value: 51.06586101023988
- type: nauc_mrr_at_10_std
value: 4.073782411624432
- type: nauc_mrr_at_1_diff1
value: 78.50105177648346
- type: nauc_mrr_at_1_max
value: 47.248470892515684
- type: nauc_mrr_at_1_std
value: -0.11060813067222093
- type: nauc_mrr_at_20_diff1
value: 77.82129736699977
- type: nauc_mrr_at_20_max
value: 51.012937537438276
- type: nauc_mrr_at_20_std
value: 4.026750215273545
- type: nauc_mrr_at_3_diff1
value: 77.56609810742842
- type: nauc_mrr_at_3_max
value: 51.08792135745153
- type: nauc_mrr_at_3_std
value: 3.7554418654239194
- type: nauc_mrr_at_5_diff1
value: 77.7457399545338
- type: nauc_mrr_at_5_max
value: 51.16775608390308
- type: nauc_mrr_at_5_std
value: 3.9707838381187424
- type: nauc_ndcg_at_1000_diff1
value: 31.17446869182052
- type: nauc_ndcg_at_1000_max
value: 31.825861543141787
- type: nauc_ndcg_at_1000_std
value: 12.166423635528758
- type: nauc_ndcg_at_100_diff1
value: 30.589419495658166
- type: nauc_ndcg_at_100_max
value: 31.55901293973185
- type: nauc_ndcg_at_100_std
value: 12.004021043334799
- type: nauc_ndcg_at_10_diff1
value: 31.217352098004675
- type: nauc_ndcg_at_10_max
value: 31.43276534877207
- type: nauc_ndcg_at_10_std
value: 9.74254567600979
- type: nauc_ndcg_at_1_diff1
value: 78.50105177648346
- type: nauc_ndcg_at_1_max
value: 47.248470892515684
- type: nauc_ndcg_at_1_std
value: -0.11060813067222093
- type: nauc_ndcg_at_20_diff1
value: 30.63955009545761
- type: nauc_ndcg_at_20_max
value: 31.407900909815595
- type: nauc_ndcg_at_20_std
value: 10.675734441679348
- type: nauc_ndcg_at_3_diff1
value: 33.7900810265147
- type: nauc_ndcg_at_3_max
value: 31.747110649279993
- type: nauc_ndcg_at_3_std
value: 7.155556583622552
- type: nauc_ndcg_at_5_diff1
value: 32.10321602077098
- type: nauc_ndcg_at_5_max
value: 31.732483389871547
- type: nauc_ndcg_at_5_std
value: 8.614224406892642
- type: nauc_precision_at_1000_diff1
value: -4.3771895114465735
- type: nauc_precision_at_1000_max
value: 18.80939383634996
- type: nauc_precision_at_1000_std
value: 35.38300059546089
- type: nauc_precision_at_100_diff1
value: 2.0633300518126068
- type: nauc_precision_at_100_max
value: 20.488227288182774
- type: nauc_precision_at_100_std
value: 26.494486955878
- type: nauc_precision_at_10_diff1
value: 12.678063050960755
- type: nauc_precision_at_10_max
value: 24.01939486753801
- type: nauc_precision_at_10_std
value: 14.77454841034483
- type: nauc_precision_at_1_diff1
value: 78.50105177648346
- type: nauc_precision_at_1_max
value: 47.248470892515684
- type: nauc_precision_at_1_std
value: -0.11060813067222093
- type: nauc_precision_at_20_diff1
value: 8.35518929563371
- type: nauc_precision_at_20_max
value: 22.43546646048988
- type: nauc_precision_at_20_std
value: 17.967498520879293
- type: nauc_precision_at_3_diff1
value: 20.744322642049422
- type: nauc_precision_at_3_max
value: 26.787609470814633
- type: nauc_precision_at_3_std
value: 9.399033393142906
- type: nauc_precision_at_5_diff1
value: 16.208995977080793
- type: nauc_precision_at_5_max
value: 25.614946548940853
- type: nauc_precision_at_5_std
value: 11.95867415429728
- type: nauc_recall_at_1000_diff1
value: -4.3771895114463595
- type: nauc_recall_at_1000_max
value: 18.80939383635008
- type: nauc_recall_at_1000_std
value: 35.38300059546097
- type: nauc_recall_at_100_diff1
value: 2.0633300518125637
- type: nauc_recall_at_100_max
value: 20.48822728818252
- type: nauc_recall_at_100_std
value: 26.4944869558779
- type: nauc_recall_at_10_diff1
value: 12.67806305096069
- type: nauc_recall_at_10_max
value: 24.019394867537958
- type: nauc_recall_at_10_std
value: 14.774548410344776
- type: nauc_recall_at_1_diff1
value: 78.50105177648346
- type: nauc_recall_at_1_max
value: 47.248470892515684
- type: nauc_recall_at_1_std
value: -0.11060813067222093
- type: nauc_recall_at_20_diff1
value: 8.35518929563362
- type: nauc_recall_at_20_max
value: 22.435466460489813
- type: nauc_recall_at_20_std
value: 17.967498520879364
- type: nauc_recall_at_3_diff1
value: 20.744322642049454
- type: nauc_recall_at_3_max
value: 26.787609470814644
- type: nauc_recall_at_3_std
value: 9.399033393142856
- type: nauc_recall_at_5_diff1
value: 16.208995977080782
- type: nauc_recall_at_5_max
value: 25.614946548940825
- type: nauc_recall_at_5_std
value: 11.958674154297341
- type: ndcg_at_1
value: 78.852
- type: ndcg_at_10
value: 67.472
- type: ndcg_at_100
value: 70.587
- type: ndcg_at_1000
value: 71.7
- type: ndcg_at_20
value: 68.852
- type: ndcg_at_3
value: 62.796
- type: ndcg_at_5
value: 65.464
- type: precision_at_1
value: 78.852
- type: precision_at_10
value: 13.901
- type: precision_at_100
value: 1.6310000000000002
- type: precision_at_1000
value: 0.178
- type: precision_at_20
value: 7.393
- type: precision_at_3
value: 39.473
- type: precision_at_5
value: 25.788
- type: recall_at_1
value: 39.426
- type: recall_at_10
value: 69.50699999999999
- type: recall_at_100
value: 81.526
- type: recall_at_1000
value: 88.90599999999999
- type: recall_at_20
value: 73.92999999999999
- type: recall_at_3
value: 59.209999999999994
- type: recall_at_5
value: 64.47
task:
type: Retrieval
- dataset:
config: default
name: MTEB MMarcoRetrieval (default)
revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
split: test
type: C-MTEB/MMarcoRetrieval
metrics:
- type: main_score
value: 57.19499999999999
- type: map_at_1
value: 44.818999999999996
- type: map_at_10
value: 53.224000000000004
- type: map_at_100
value: 53.757
- type: map_at_1000
value: 53.788000000000004
- type: map_at_20
value: 53.561
- type: map_at_3
value: 51.188
- type: map_at_5
value: 52.443
- type: mrr_at_1
value: 46.41833810888252
- type: mrr_at_10
value: 53.94587142402337
- type: mrr_at_100
value: 54.43571159357003
- type: mrr_at_1000
value: 54.465100780371344
- type: mrr_at_20
value: 54.257204411188184
- type: mrr_at_3
value: 52.101241642789105
- type: mrr_at_5
value: 53.233763132760345
- type: nauc_map_at_1000_diff1
value: 70.26042979905807
- type: nauc_map_at_1000_max
value: 58.68416950903844
- type: nauc_map_at_1000_std
value: 13.410551528553881
- type: nauc_map_at_100_diff1
value: 70.25125347260497
- type: nauc_map_at_100_max
value: 58.69264269006522
- type: nauc_map_at_100_std
value: 13.42088619384096
- type: nauc_map_at_10_diff1
value: 70.24781652539009
- type: nauc_map_at_10_max
value: 58.66302990334148
- type: nauc_map_at_10_std
value: 13.107489577012036
- type: nauc_map_at_1_diff1
value: 74.03972607662418
- type: nauc_map_at_1_max
value: 52.9732154435787
- type: nauc_map_at_1_std
value: 8.560702338785676
- type: nauc_map_at_20_diff1
value: 70.24288756743213
- type: nauc_map_at_20_max
value: 58.74172597665706
- type: nauc_map_at_20_std
value: 13.34597982289252
- type: nauc_map_at_3_diff1
value: 70.8300198090176
- type: nauc_map_at_3_max
value: 57.290560592566294
- type: nauc_map_at_3_std
value: 11.577199426192937
- type: nauc_map_at_5_diff1
value: 70.43174453367
- type: nauc_map_at_5_max
value: 58.298993661414734
- type: nauc_map_at_5_std
value: 12.575445542352456
- type: nauc_mrr_at_1000_diff1
value: 70.59310122342762
- type: nauc_mrr_at_1000_max
value: 59.20680864869718
- type: nauc_mrr_at_1000_std
value: 13.933431988401148
- type: nauc_mrr_at_100_diff1
value: 70.58484769325997
- type: nauc_mrr_at_100_max
value: 59.21586893327749
- type: nauc_mrr_at_100_std
value: 13.945168626665009
- type: nauc_mrr_at_10_diff1
value: 70.56835533190241
- type: nauc_mrr_at_10_max
value: 59.20304563671691
- type: nauc_mrr_at_10_std
value: 13.705059776100947
- type: nauc_mrr_at_1_diff1
value: 74.70229476795922
- type: nauc_mrr_at_1_max
value: 55.24461562825211
- type: nauc_mrr_at_1_std
value: 9.388508061475354
- type: nauc_mrr_at_20_diff1
value: 70.57379668885889
- type: nauc_mrr_at_20_max
value: 59.261650643021554
- type: nauc_mrr_at_20_std
value: 13.892150772800923
- type: nauc_mrr_at_3_diff1
value: 71.11578189201518
- type: nauc_mrr_at_3_max
value: 58.18069865079906
- type: nauc_mrr_at_3_std
value: 12.450376884835663
- type: nauc_mrr_at_5_diff1
value: 70.72202593045206
- type: nauc_mrr_at_5_max
value: 58.97723226815985
- type: nauc_mrr_at_5_std
value: 13.260634760348719
- type: nauc_ndcg_at_1000_diff1
value: 68.74703378326001
- type: nauc_ndcg_at_1000_max
value: 60.83223290736585
- type: nauc_ndcg_at_1000_std
value: 16.981502750506138
- type: nauc_ndcg_at_100_diff1
value: 68.47529577711941
- type: nauc_ndcg_at_100_max
value: 61.091478450501754
- type: nauc_ndcg_at_100_std
value: 17.483370931736314
- type: nauc_ndcg_at_10_diff1
value: 68.50930607099632
- type: nauc_ndcg_at_10_max
value: 61.10969456714638
- type: nauc_ndcg_at_10_std
value: 15.936588003469698
- type: nauc_ndcg_at_1_diff1
value: 74.70229476795922
- type: nauc_ndcg_at_1_max
value: 55.24461562825211
- type: nauc_ndcg_at_1_std
value: 9.388508061475354
- type: nauc_ndcg_at_20_diff1
value: 68.46317512015115
- type: nauc_ndcg_at_20_max
value: 61.41365545045488
- type: nauc_ndcg_at_20_std
value: 16.844459770935778
- type: nauc_ndcg_at_3_diff1
value: 69.74824086530852
- type: nauc_ndcg_at_3_max
value: 58.57331207942189
- type: nauc_ndcg_at_3_std
value: 12.897388629534762
- type: nauc_ndcg_at_5_diff1
value: 68.97682800960776
- type: nauc_ndcg_at_5_max
value: 60.299098240569705
- type: nauc_ndcg_at_5_std
value: 14.632067497966005
- type: nauc_precision_at_1000_diff1
value: 26.89007113883275
- type: nauc_precision_at_1000_max
value: 46.23130390879421
- type: nauc_precision_at_1000_std
value: 30.698138975519285
- type: nauc_precision_at_100_diff1
value: 35.93712021176337
- type: nauc_precision_at_100_max
value: 54.76852109365306
- type: nauc_precision_at_100_std
value: 33.907696965402295
- type: nauc_precision_at_10_diff1
value: 48.371135821722596
- type: nauc_precision_at_10_max
value: 61.99559796089503
- type: nauc_precision_at_10_std
value: 24.754732150638066
- type: nauc_precision_at_1_diff1
value: 74.70229476795922
- type: nauc_precision_at_1_max
value: 55.24461562825211
- type: nauc_precision_at_1_std
value: 9.388508061475354
- type: nauc_precision_at_20_diff1
value: 44.15746389562522
- type: nauc_precision_at_20_max
value: 61.10027662732338
- type: nauc_precision_at_20_std
value: 28.858378094065635
- type: nauc_precision_at_3_diff1
value: 59.33964667148327
- type: nauc_precision_at_3_max
value: 59.660666754413285
- type: nauc_precision_at_3_std
value: 16.30519160170101
- type: nauc_precision_at_5_diff1
value: 54.065633958559225
- type: nauc_precision_at_5_max
value: 61.842642200679634
- type: nauc_precision_at_5_std
value: 20.633161696275206
- type: nauc_recall_at_1000_diff1
value: 58.1619901023936
- type: nauc_recall_at_1000_max
value: 72.76628545759259
- type: nauc_recall_at_1000_std
value: 43.07750525698423
- type: nauc_recall_at_100_diff1
value: 57.81340416728631
- type: nauc_recall_at_100_max
value: 71.865265055528
- type: nauc_recall_at_100_std
value: 41.445617320943356
- type: nauc_recall_at_10_diff1
value: 61.354501272492165
- type: nauc_recall_at_10_max
value: 69.31314165032313
- type: nauc_recall_at_10_std
value: 25.94549688317928
- type: nauc_recall_at_1_diff1
value: 74.03972607662418
- type: nauc_recall_at_1_max
value: 52.9732154435787
- type: nauc_recall_at_1_std
value: 8.560702338785676
- type: nauc_recall_at_20_diff1
value: 60.1168579200214
- type: nauc_recall_at_20_max
value: 71.73426250754902
- type: nauc_recall_at_20_std
value: 31.743125064742
- type: nauc_recall_at_3_diff1
value: 65.99302068813246
- type: nauc_recall_at_3_max
value: 61.38084814191331
- type: nauc_recall_at_3_std
value: 15.986612831059693
- type: nauc_recall_at_5_diff1
value: 63.442426874234826
- type: nauc_recall_at_5_max
value: 65.83283626666224
- type: nauc_recall_at_5_std
value: 20.675189002245734
- type: ndcg_at_1
value: 46.418
- type: ndcg_at_10
value: 57.19499999999999
- type: ndcg_at_100
value: 59.73500000000001
- type: ndcg_at_1000
value: 60.633
- type: ndcg_at_20
value: 58.367999999999995
- type: ndcg_at_3
value: 53.217999999999996
- type: ndcg_at_5
value: 55.382
- type: precision_at_1
value: 46.418
- type: precision_at_10
value: 7.317
- type: precision_at_100
value: 0.86
- type: precision_at_1000
value: 0.094
- type: precision_at_20
value: 3.9050000000000002
- type: precision_at_3
value: 20.463
- type: precision_at_5
value: 13.421
- type: recall_at_1
value: 44.818999999999996
- type: recall_at_10
value: 68.829
- type: recall_at_100
value: 80.60199999999999
- type: recall_at_1000
value: 87.75500000000001
- type: recall_at_20
value: 73.355
- type: recall_at_3
value: 58.201
- type: recall_at_5
value: 63.354
task:
type: Retrieval
- dataset:
config: default
name: MTEB MSMARCO (default)
revision: c5a29a104738b98a9e76336939199e264163d4a0
split: dev
type: mteb/msmarco
metrics:
- type: main_score
value: 17.108
- type: map_at_1
value: 8.524
- type: map_at_10
value: 13.811000000000002
- type: map_at_100
value: 14.693000000000001
- type: map_at_1000
value: 14.77
- type: map_at_20
value: 14.319
- type: map_at_3
value: 11.892999999999999
- type: map_at_5
value: 12.922
- type: mrr_at_1
value: 8.796561604584527
- type: mrr_at_10
value: 14.170407513530764
- type: mrr_at_100
value: 15.048561930734833
- type: mrr_at_1000
value: 15.122376489953226
- type: mrr_at_20
value: 14.681627161807759
- type: mrr_at_3
value: 12.261222540592206
- type: mrr_at_5
value: 13.289159503342962
- type: nauc_map_at_1000_diff1
value: 23.218668478025386
- type: nauc_map_at_1000_max
value: 29.61958688093464
- type: nauc_map_at_1000_std
value: 11.547786129171026
- type: nauc_map_at_100_diff1
value: 23.223024602139336
- type: nauc_map_at_100_max
value: 29.569381710107816
- type: nauc_map_at_100_std
value: 11.463448202909808
- type: nauc_map_at_10_diff1
value: 23.42245083616564
- type: nauc_map_at_10_max
value: 28.736145176842225
- type: nauc_map_at_10_std
value: 9.905604720875653
- type: nauc_map_at_1_diff1
value: 29.8748627435949
- type: nauc_map_at_1_max
value: 26.6497938053676
- type: nauc_map_at_1_std
value: 6.121872414111879
- type: nauc_map_at_20_diff1
value: 23.361862479281662
- type: nauc_map_at_20_max
value: 29.231361636696505
- type: nauc_map_at_20_std
value: 10.801655860954275
- type: nauc_map_at_3_diff1
value: 24.743952241108826
- type: nauc_map_at_3_max
value: 27.472041480944654
- type: nauc_map_at_3_std
value: 8.004052762828538
- type: nauc_map_at_5_diff1
value: 23.779734368818893
- type: nauc_map_at_5_max
value: 27.915554800469938
- type: nauc_map_at_5_std
value: 8.739318343988039
- type: nauc_mrr_at_1000_diff1
value: 23.008369224805534
- type: nauc_mrr_at_1000_max
value: 29.565473885504773
- type: nauc_mrr_at_1000_std
value: 11.847071636350016
- type: nauc_mrr_at_100_diff1
value: 23.008284426027256
- type: nauc_mrr_at_100_max
value: 29.518921482024613
- type: nauc_mrr_at_100_std
value: 11.775847622511266
- type: nauc_mrr_at_10_diff1
value: 23.195952911900232
- type: nauc_mrr_at_10_max
value: 28.73909541090732
- type: nauc_mrr_at_10_std
value: 10.291592894731307
- type: nauc_mrr_at_1_diff1
value: 29.588774232229113
- type: nauc_mrr_at_1_max
value: 26.734307006971754
- type: nauc_mrr_at_1_std
value: 6.570898688940385
- type: nauc_mrr_at_20_diff1
value: 23.136714296525152
- type: nauc_mrr_at_20_max
value: 29.21056314342999
- type: nauc_mrr_at_20_std
value: 11.15933374731932
- type: nauc_mrr_at_3_diff1
value: 24.391945382771883
- type: nauc_mrr_at_3_max
value: 27.48307535907865
- type: nauc_mrr_at_3_std
value: 8.428671151539316
- type: nauc_mrr_at_5_diff1
value: 23.49504304491056
- type: nauc_mrr_at_5_max
value: 27.946063332613296
- type: nauc_mrr_at_5_std
value: 9.16414791963959
- type: nauc_ndcg_at_1000_diff1
value: 20.315657936642616
- type: nauc_ndcg_at_1000_max
value: 34.617184131903876
- type: nauc_ndcg_at_1000_std
value: 20.642322084890235
- type: nauc_ndcg_at_100_diff1
value: 20.336347873884456
- type: nauc_ndcg_at_100_max
value: 33.472006688498624
- type: nauc_ndcg_at_100_std
value: 18.922299187193342
- type: nauc_ndcg_at_10_diff1
value: 21.298008062935423
- type: nauc_ndcg_at_10_max
value: 30.066919227586975
- type: nauc_ndcg_at_10_std
value: 12.160605785363709
- type: nauc_ndcg_at_1_diff1
value: 29.588774232229113
- type: nauc_ndcg_at_1_max
value: 26.734307006971754
- type: nauc_ndcg_at_1_std
value: 6.570898688940385
- type: nauc_ndcg_at_20_diff1
value: 21.161652731769628
- type: nauc_ndcg_at_20_max
value: 31.574315289059758
- type: nauc_ndcg_at_20_std
value: 14.910044538877084
- type: nauc_ndcg_at_3_diff1
value: 23.33332537198534
- type: nauc_ndcg_at_3_max
value: 27.684419184625824
- type: nauc_ndcg_at_3_std
value: 8.607652342532672
- type: nauc_ndcg_at_5_diff1
value: 21.91785854168375
- type: nauc_ndcg_at_5_max
value: 28.358978848989754
- type: nauc_ndcg_at_5_std
value: 9.759960992476355
- type: nauc_precision_at_1000_diff1
value: 10.365579440883709
- type: nauc_precision_at_1000_max
value: 47.8276720411195
- type: nauc_precision_at_1000_std
value: 50.25815541266756
- type: nauc_precision_at_100_diff1
value: 13.449076303656716
- type: nauc_precision_at_100_max
value: 42.29622038814561
- type: nauc_precision_at_100_std
value: 38.32848146914202
- type: nauc_precision_at_10_diff1
value: 16.817806945711304
- type: nauc_precision_at_10_max
value: 32.65098242638481
- type: nauc_precision_at_10_std
value: 17.1057301972624
- type: nauc_precision_at_1_diff1
value: 29.588774232229113
- type: nauc_precision_at_1_max
value: 26.734307006971754
- type: nauc_precision_at_1_std
value: 6.570898688940385
- type: nauc_precision_at_20_diff1
value: 16.6325483032766
- type: nauc_precision_at_20_max
value: 36.4739965581178
- type: nauc_precision_at_20_std
value: 24.243947927232494
- type: nauc_precision_at_3_diff1
value: 20.03994877797651
- type: nauc_precision_at_3_max
value: 28.001992724251544
- type: nauc_precision_at_3_std
value: 9.918910685983798
- type: nauc_precision_at_5_diff1
value: 17.909288257232078
- type: nauc_precision_at_5_max
value: 29.440573068780562
- type: nauc_precision_at_5_std
value: 12.138201555824145
- type: nauc_recall_at_1000_diff1
value: 12.384404623990866
- type: nauc_recall_at_1000_max
value: 53.16894570389471
- type: nauc_recall_at_1000_std
value: 54.0339774253684
- type: nauc_recall_at_100_diff1
value: 13.916689646493092
- type: nauc_recall_at_100_max
value: 43.29116542457196
- type: nauc_recall_at_100_std
value: 38.38534819483866
- type: nauc_recall_at_10_diff1
value: 17.167697112334153
- type: nauc_recall_at_10_max
value: 32.72836401992213
- type: nauc_recall_at_10_std
value: 16.586757497950337
- type: nauc_recall_at_1_diff1
value: 29.8748627435949
- type: nauc_recall_at_1_max
value: 26.6497938053676
- type: nauc_recall_at_1_std
value: 6.121872414111879
- type: nauc_recall_at_20_diff1
value: 16.995799057981877
- type: nauc_recall_at_20_max
value: 36.50375112670718
- type: nauc_recall_at_20_std
value: 23.6626167330791
- type: nauc_recall_at_3_diff1
value: 20.335741054398525
- type: nauc_recall_at_3_max
value: 28.041771518841628
- type: nauc_recall_at_3_std
value: 9.630422694689473
- type: nauc_recall_at_5_diff1
value: 18.029821500684772
- type: nauc_recall_at_5_max
value: 28.9177721372625
- type: nauc_recall_at_5_std
value: 11.484499860649166
- type: ndcg_at_1
value: 8.797
- type: ndcg_at_10
value: 17.108
- type: ndcg_at_100
value: 21.684
- type: ndcg_at_1000
value: 23.929000000000002
- type: ndcg_at_20
value: 18.964
- type: ndcg_at_3
value: 13.128
- type: ndcg_at_5
value: 14.972
- type: precision_at_1
value: 8.797
- type: precision_at_10
value: 2.858
- type: precision_at_100
value: 0.518
- type: precision_at_1000
value: 0.07100000000000001
- type: precision_at_20
value: 1.814
- type: precision_at_3
value: 5.669
- type: precision_at_5
value: 4.324
- type: recall_at_1
value: 8.524
- type: recall_at_10
value: 27.303
- type: recall_at_100
value: 49.099
- type: recall_at_1000
value: 66.988
- type: recall_at_20
value: 34.571000000000005
- type: recall_at_3
value: 16.351
- type: recall_at_5
value: 20.79
task:
type: Retrieval
- dataset:
config: default
name: MTEB MedicalRetrieval (default)
revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
split: test
type: C-MTEB/MedicalRetrieval
metrics:
- type: main_score
value: 42.986000000000004
- type: map_at_1
value: 36.5
- type: map_at_10
value: 40.871
- type: map_at_100
value: 41.386
- type: map_at_1000
value: 41.435
- type: map_at_20
value: 41.149
- type: map_at_3
value: 39.933
- type: map_at_5
value: 40.348
- type: mrr_at_1
value: 36.7
- type: mrr_at_10
value: 40.97107142857142
- type: mrr_at_100
value: 41.48627116126376
- type: mrr_at_1000
value: 41.5350651501168
- type: mrr_at_20
value: 41.248736283925915
- type: mrr_at_3
value: 40.03333333333332
- type: mrr_at_5
value: 40.448333333333316
- type: nauc_map_at_1000_diff1
value: 77.37512949727217
- type: nauc_map_at_1000_max
value: 72.06477471911317
- type: nauc_map_at_1000_std
value: 19.094217395528524
- type: nauc_map_at_100_diff1
value: 77.373933280668
- type: nauc_map_at_100_max
value: 72.0649042565364
- type: nauc_map_at_100_std
value: 19.083791137816352
- type: nauc_map_at_10_diff1
value: 77.70989297249345
- type: nauc_map_at_10_max
value: 72.27495008631554
- type: nauc_map_at_10_std
value: 18.932224004065162
- type: nauc_map_at_1_diff1
value: 82.52567232363161
- type: nauc_map_at_1_max
value: 74.55360590610704
- type: nauc_map_at_1_std
value: 20.515475387948243
- type: nauc_map_at_20_diff1
value: 77.46838293283584
- type: nauc_map_at_20_max
value: 72.14297394136004
- type: nauc_map_at_20_std
value: 19.02131554485847
- type: nauc_map_at_3_diff1
value: 78.35949763496411
- type: nauc_map_at_3_max
value: 72.68798107449088
- type: nauc_map_at_3_std
value: 18.99379028492272
- type: nauc_map_at_5_diff1
value: 78.29663044814824
- type: nauc_map_at_5_max
value: 72.85092001611716
- type: nauc_map_at_5_std
value: 19.09987622335474
- type: nauc_mrr_at_1000_diff1
value: 77.0556567524976
- type: nauc_mrr_at_1000_max
value: 72.09048487668457
- type: nauc_mrr_at_1000_std
value: 19.25214135903195
- type: nauc_mrr_at_100_diff1
value: 77.05472401560309
- type: nauc_mrr_at_100_max
value: 72.09059311719659
- type: nauc_mrr_at_100_std
value: 19.241590200834146
- type: nauc_mrr_at_10_diff1
value: 77.39213531475731
- type: nauc_mrr_at_10_max
value: 72.30044601578872
- type: nauc_mrr_at_10_std
value: 19.08939059922627
- type: nauc_mrr_at_1_diff1
value: 81.8862308729431
- type: nauc_mrr_at_1_max
value: 74.60235468275339
- type: nauc_mrr_at_1_std
value: 20.8279444323117
- type: nauc_mrr_at_20_diff1
value: 77.14991068904308
- type: nauc_mrr_at_20_max
value: 72.16860011690952
- type: nauc_mrr_at_20_std
value: 19.178850208436376
- type: nauc_mrr_at_3_diff1
value: 78.04245151703907
- type: nauc_mrr_at_3_max
value: 72.71319714374202
- type: nauc_mrr_at_3_std
value: 19.150421583399723
- type: nauc_mrr_at_5_diff1
value: 77.97916227480808
- type: nauc_mrr_at_5_max
value: 72.87609390630256
- type: nauc_mrr_at_5_std
value: 19.256680048263735
- type: nauc_ndcg_at_1000_diff1
value: 73.93767113477907
- type: nauc_ndcg_at_1000_max
value: 69.88404116733506
- type: nauc_ndcg_at_1000_std
value: 19.68407713864583
- type: nauc_ndcg_at_100_diff1
value: 73.82395257759265
- type: nauc_ndcg_at_100_max
value: 69.83618991706564
- type: nauc_ndcg_at_100_std
value: 19.256279161584654
- type: nauc_ndcg_at_10_diff1
value: 75.58036680360185
- type: nauc_ndcg_at_10_max
value: 70.9277731951102
- type: nauc_ndcg_at_10_std
value: 18.24765302865303
- type: nauc_ndcg_at_1_diff1
value: 82.52567232363161
- type: nauc_ndcg_at_1_max
value: 74.55360590610704
- type: nauc_ndcg_at_1_std
value: 20.515475387948243
- type: nauc_ndcg_at_20_diff1
value: 74.67267264640863
- type: nauc_ndcg_at_20_max
value: 70.41638512690534
- type: nauc_ndcg_at_20_std
value: 18.5318975733297
- type: nauc_ndcg_at_3_diff1
value: 77.1329446571036
- type: nauc_ndcg_at_3_max
value: 72.08931244236261
- type: nauc_ndcg_at_3_std
value: 18.53107234264309
- type: nauc_ndcg_at_5_diff1
value: 77.01484051719324
- type: nauc_ndcg_at_5_max
value: 72.35261224856936
- type: nauc_ndcg_at_5_std
value: 18.72538724810479
- type: nauc_precision_at_1000_diff1
value: 53.30172161915614
- type: nauc_precision_at_1000_max
value: 56.461034398927545
- type: nauc_precision_at_1000_std
value: 28.81119286639875
- type: nauc_precision_at_100_diff1
value: 57.78206578042088
- type: nauc_precision_at_100_max
value: 59.606013396493516
- type: nauc_precision_at_100_std
value: 21.912993966042226
- type: nauc_precision_at_10_diff1
value: 68.76288618627447
- type: nauc_precision_at_10_max
value: 66.39389103261195
- type: nauc_precision_at_10_std
value: 16.014609557706812
- type: nauc_precision_at_1_diff1
value: 82.52567232363161
- type: nauc_precision_at_1_max
value: 74.55360590610704
- type: nauc_precision_at_1_std
value: 20.515475387948243
- type: nauc_precision_at_20_diff1
value: 64.74676681608756
- type: nauc_precision_at_20_max
value: 64.07208764182178
- type: nauc_precision_at_20_std
value: 17.01363067962291
- type: nauc_precision_at_3_diff1
value: 73.62522389481089
- type: nauc_precision_at_3_max
value: 70.36056685665382
- type: nauc_precision_at_3_std
value: 17.202818367658427
- type: nauc_precision_at_5_diff1
value: 73.31220817816539
- type: nauc_precision_at_5_max
value: 70.92407676989903
- type: nauc_precision_at_5_std
value: 17.67739682781376
- type: nauc_recall_at_1000_diff1
value: 53.301721619156105
- type: nauc_recall_at_1000_max
value: 56.461034398927666
- type: nauc_recall_at_1000_std
value: 28.811192866398837
- type: nauc_recall_at_100_diff1
value: 57.782065780420844
- type: nauc_recall_at_100_max
value: 59.60601339649349
- type: nauc_recall_at_100_std
value: 21.912993966042183
- type: nauc_recall_at_10_diff1
value: 68.7628861862745
- type: nauc_recall_at_10_max
value: 66.39389103261195
- type: nauc_recall_at_10_std
value: 16.014609557706795
- type: nauc_recall_at_1_diff1
value: 82.52567232363161
- type: nauc_recall_at_1_max
value: 74.55360590610704
- type: nauc_recall_at_1_std
value: 20.515475387948243
- type: nauc_recall_at_20_diff1
value: 64.74676681608761
- type: nauc_recall_at_20_max
value: 64.07208764182177
- type: nauc_recall_at_20_std
value: 17.013630679622935
- type: nauc_recall_at_3_diff1
value: 73.62522389481089
- type: nauc_recall_at_3_max
value: 70.36056685665386
- type: nauc_recall_at_3_std
value: 17.202818367658445
- type: nauc_recall_at_5_diff1
value: 73.3122081781654
- type: nauc_recall_at_5_max
value: 70.92407676989899
- type: nauc_recall_at_5_std
value: 17.677396827813805
- type: ndcg_at_1
value: 36.5
- type: ndcg_at_10
value: 42.986000000000004
- type: ndcg_at_100
value: 45.79
- type: ndcg_at_1000
value: 47.407
- type: ndcg_at_20
value: 43.998
- type: ndcg_at_3
value: 40.959
- type: ndcg_at_5
value: 41.704
- type: precision_at_1
value: 36.5
- type: precision_at_10
value: 4.97
- type: precision_at_100
value: 0.635
- type: precision_at_1000
value: 0.077
- type: precision_at_20
value: 2.685
- type: precision_at_3
value: 14.633
- type: precision_at_5
value: 9.139999999999999
- type: recall_at_1
value: 36.5
- type: recall_at_10
value: 49.7
- type: recall_at_100
value: 63.5
- type: recall_at_1000
value: 76.9
- type: recall_at_20
value: 53.7
- type: recall_at_3
value: 43.9
- type: recall_at_5
value: 45.7
task:
type: Retrieval
- dataset:
config: default
name: MTEB NFCorpus (default)
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
split: test
type: mteb/nfcorpus
metrics:
- type: main_score
value: 28.717
- type: map_at_1
value: 5.1339999999999995
- type: map_at_10
value: 11.116
- type: map_at_100
value: 12.909
- type: map_at_1000
value: 13.514000000000001
- type: map_at_20
value: 12.017
- type: map_at_3
value: 8.530999999999999
- type: map_at_5
value: 9.849
- type: mrr_at_1
value: 42.10526315789473
- type: mrr_at_10
value: 48.94896555113274
- type: mrr_at_100
value: 49.538778225230004
- type: mrr_at_1000
value: 49.5972692996884
- type: mrr_at_20
value: 49.33970910655248
- type: mrr_at_3
value: 47.36842105263159
- type: mrr_at_5
value: 48.28173374613004
- type: nauc_map_at_1000_diff1
value: 16.353893868263473
- type: nauc_map_at_1000_max
value: 44.040542152095554
- type: nauc_map_at_1000_std
value: 18.193794214316128
- type: nauc_map_at_100_diff1
value: 16.74853812715935
- type: nauc_map_at_100_max
value: 43.99843857525908
- type: nauc_map_at_100_std
value: 17.027768108245347
- type: nauc_map_at_10_diff1
value: 20.518880945162515
- type: nauc_map_at_10_max
value: 41.97522979226856
- type: nauc_map_at_10_std
value: 11.82591379526025
- type: nauc_map_at_1_diff1
value: 40.60820182767276
- type: nauc_map_at_1_max
value: 34.324314249089035
- type: nauc_map_at_1_std
value: 6.241429658500057
- type: nauc_map_at_20_diff1
value: 18.029190338253287
- type: nauc_map_at_20_max
value: 43.59909950180384
- type: nauc_map_at_20_std
value: 14.392341317877039
- type: nauc_map_at_3_diff1
value: 26.151377515810985
- type: nauc_map_at_3_max
value: 37.24496964175979
- type: nauc_map_at_3_std
value: 7.744086080739083
- type: nauc_map_at_5_diff1
value: 24.727410429739503
- type: nauc_map_at_5_max
value: 38.995534077321494
- type: nauc_map_at_5_std
value: 8.94368246607561
- type: nauc_mrr_at_1000_diff1
value: 28.912407160224873
- type: nauc_mrr_at_1000_max
value: 52.20648880316783
- type: nauc_mrr_at_1000_std
value: 18.67711355101332
- type: nauc_mrr_at_100_diff1
value: 28.862722538310585
- type: nauc_mrr_at_100_max
value: 52.21237555020183
- type: nauc_mrr_at_100_std
value: 18.711444597208224
- type: nauc_mrr_at_10_diff1
value: 29.249669752599406
- type: nauc_mrr_at_10_max
value: 52.125996129180706
- type: nauc_mrr_at_10_std
value: 18.108543640748522
- type: nauc_mrr_at_1_diff1
value: 31.437393726415657
- type: nauc_mrr_at_1_max
value: 50.30794594214224
- type: nauc_mrr_at_1_std
value: 15.797200651547834
- type: nauc_mrr_at_20_diff1
value: 28.960276249385753
- type: nauc_mrr_at_20_max
value: 52.2292512355103
- type: nauc_mrr_at_20_std
value: 18.647209256171255
- type: nauc_mrr_at_3_diff1
value: 29.9403099791075
- type: nauc_mrr_at_3_max
value: 52.21375418646459
- type: nauc_mrr_at_3_std
value: 17.657403630905236
- type: nauc_mrr_at_5_diff1
value: 29.288691954701605
- type: nauc_mrr_at_5_max
value: 51.93458325391017
- type: nauc_mrr_at_5_std
value: 17.449275114151224
- type: nauc_ndcg_at_1000_diff1
value: 17.413449466355477
- type: nauc_ndcg_at_1000_max
value: 51.282128853882966
- type: nauc_ndcg_at_1000_std
value: 26.676699834172833
- type: nauc_ndcg_at_100_diff1
value: 14.229405037669823
- type: nauc_ndcg_at_100_max
value: 47.27756231700748
- type: nauc_ndcg_at_100_std
value: 26.359853706665675
- type: nauc_ndcg_at_10_diff1
value: 10.756820538491432
- type: nauc_ndcg_at_10_max
value: 43.25352958404275
- type: nauc_ndcg_at_10_std
value: 21.498972415819615
- type: nauc_ndcg_at_1_diff1
value: 32.08481502771464
- type: nauc_ndcg_at_1_max
value: 49.512698007983666
- type: nauc_ndcg_at_1_std
value: 15.52228503821167
- type: nauc_ndcg_at_20_diff1
value: 10.584244885381274
- type: nauc_ndcg_at_20_max
value: 43.682615411785605
- type: nauc_ndcg_at_20_std
value: 23.91850402894147
- type: nauc_ndcg_at_3_diff1
value: 14.368283308364383
- type: nauc_ndcg_at_3_max
value: 46.3845049842012
- type: nauc_ndcg_at_3_std
value: 20.362210144259294
- type: nauc_ndcg_at_5_diff1
value: 12.794798632954132
- type: nauc_ndcg_at_5_max
value: 44.81887933731769
- type: nauc_ndcg_at_5_std
value: 20.17434710663848
- type: nauc_precision_at_1000_diff1
value: -11.62222903948536
- type: nauc_precision_at_1000_max
value: 3.85955586789508
- type: nauc_precision_at_1000_std
value: 27.242984511883318
- type: nauc_precision_at_100_diff1
value: -15.235363834286758
- type: nauc_precision_at_100_max
value: 15.552980262829665
- type: nauc_precision_at_100_std
value: 35.902059450309075
- type: nauc_precision_at_10_diff1
value: -8.569695903294772
- type: nauc_precision_at_10_max
value: 34.279516997538
- type: nauc_precision_at_10_std
value: 25.326968367382264
- type: nauc_precision_at_1_diff1
value: 31.437393726415657
- type: nauc_precision_at_1_max
value: 50.30794594214224
- type: nauc_precision_at_1_std
value: 15.797200651547834
- type: nauc_precision_at_20_diff1
value: -13.581809281052983
- type: nauc_precision_at_20_max
value: 28.36321164223102
- type: nauc_precision_at_20_std
value: 29.30831094247851
- type: nauc_precision_at_3_diff1
value: 4.528067814541078
- type: nauc_precision_at_3_max
value: 43.00955396308342
- type: nauc_precision_at_3_std
value: 21.09295855114464
- type: nauc_precision_at_5_diff1
value: -0.49426825672219826
- type: nauc_precision_at_5_max
value: 39.70156191883351
- type: nauc_precision_at_5_std
value: 21.671532426934093
- type: nauc_recall_at_1000_diff1
value: 14.660764095574033
- type: nauc_recall_at_1000_max
value: 35.49957428338693
- type: nauc_recall_at_1000_std
value: 19.47138042432338
- type: nauc_recall_at_100_diff1
value: 10.252566830606682
- type: nauc_recall_at_100_max
value: 40.61686153859631
- type: nauc_recall_at_100_std
value: 23.545459884553615
- type: nauc_recall_at_10_diff1
value: 18.740737180025736
- type: nauc_recall_at_10_max
value: 38.69765703939695
- type: nauc_recall_at_10_std
value: 10.071092204381513
- type: nauc_recall_at_1_diff1
value: 40.60820182767276
- type: nauc_recall_at_1_max
value: 34.324314249089035
- type: nauc_recall_at_1_std
value: 6.241429658500057
- type: nauc_recall_at_20_diff1
value: 15.163058894173126
- type: nauc_recall_at_20_max
value: 39.320481512988756
- type: nauc_recall_at_20_std
value: 13.65010053759418
- type: nauc_recall_at_3_diff1
value: 24.495065506876614
- type: nauc_recall_at_3_max
value: 35.34822128689044
- type: nauc_recall_at_3_std
value: 6.2632503472831145
- type: nauc_recall_at_5_diff1
value: 22.51780662380844
- type: nauc_recall_at_5_max
value: 34.69191236760095
- type: nauc_recall_at_5_std
value: 6.71655205068772
- type: ndcg_at_1
value: 39.474
- type: ndcg_at_10
value: 28.717
- type: ndcg_at_100
value: 24.017
- type: ndcg_at_1000
value: 29.933
- type: ndcg_at_20
value: 26.183
- type: ndcg_at_3
value: 34.52
- type: ndcg_at_5
value: 31.758999999999997
- type: precision_at_1
value: 42.105
- type: precision_at_10
value: 20.402
- type: precision_at_100
value: 5.483
- type: precision_at_1000
value: 1.468
- type: precision_at_20
value: 14.474
- type: precision_at_3
value: 32.301
- type: precision_at_5
value: 26.811
- type: recall_at_1
value: 5.1339999999999995
- type: recall_at_10
value: 14.14
- type: recall_at_100
value: 21.979000000000003
- type: recall_at_1000
value: 44.57
- type: recall_at_20
value: 16.921
- type: recall_at_3
value: 9.414
- type: recall_at_5
value: 11.688
task:
type: Retrieval
- dataset:
config: default
name: MTEB NQ (default)
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
split: test
type: mteb/nq
metrics:
- type: main_score
value: 20.498
- type: map_at_1
value: 11.006
- type: map_at_10
value: 16.881
- type: map_at_100
value: 17.681
- type: map_at_1000
value: 17.747
- type: map_at_20
value: 17.349
- type: map_at_3
value: 14.782
- type: map_at_5
value: 15.986
- type: mrr_at_1
value: 12.804171494785631
- type: mrr_at_10
value: 18.50528334161012
- type: mrr_at_100
value: 19.244286714242154
- type: mrr_at_1000
value: 19.301707708547898
- type: mrr_at_20
value: 18.940757061236877
- type: mrr_at_3
value: 16.613557358053296
- type: mrr_at_5
value: 17.692641946697556
- type: nauc_map_at_1000_diff1
value: 31.665402625771303
- type: nauc_map_at_1000_max
value: 39.93401758057253
- type: nauc_map_at_1000_std
value: 28.92786289419731
- type: nauc_map_at_100_diff1
value: 31.679923811098586
- type: nauc_map_at_100_max
value: 39.940250466709145
- type: nauc_map_at_100_std
value: 28.90733543889832
- type: nauc_map_at_10_diff1
value: 31.683994809354456
- type: nauc_map_at_10_max
value: 39.38998633561208
- type: nauc_map_at_10_std
value: 27.884758211850517
- type: nauc_map_at_1_diff1
value: 37.771167547385545
- type: nauc_map_at_1_max
value: 33.79936534835281
- type: nauc_map_at_1_std
value: 19.8026545196434
- type: nauc_map_at_20_diff1
value: 31.741953072257402
- type: nauc_map_at_20_max
value: 39.78817031104777
- type: nauc_map_at_20_std
value: 28.56481501195481
- type: nauc_map_at_3_diff1
value: 32.121253875385065
- type: nauc_map_at_3_max
value: 36.73540051762868
- type: nauc_map_at_3_std
value: 24.53111302707997
- type: nauc_map_at_5_diff1
value: 31.69042846828089
- type: nauc_map_at_5_max
value: 38.38010820024265
- type: nauc_map_at_5_std
value: 26.56806841463661
- type: nauc_mrr_at_1000_diff1
value: 32.03936148344948
- type: nauc_mrr_at_1000_max
value: 40.1787187014775
- type: nauc_mrr_at_1000_std
value: 29.391476911132152
- type: nauc_mrr_at_100_diff1
value: 32.05107953270361
- type: nauc_mrr_at_100_max
value: 40.191089365197456
- type: nauc_mrr_at_100_std
value: 29.38534936580669
- type: nauc_mrr_at_10_diff1
value: 32.035247091986626
- type: nauc_mrr_at_10_max
value: 39.712324323388124
- type: nauc_mrr_at_10_std
value: 28.584921964851095
- type: nauc_mrr_at_1_diff1
value: 37.75122698222711
- type: nauc_mrr_at_1_max
value: 35.80113036602084
- type: nauc_mrr_at_1_std
value: 22.20249486383813
- type: nauc_mrr_at_20_diff1
value: 32.10750126321392
- type: nauc_mrr_at_20_max
value: 40.07242003996933
- type: nauc_mrr_at_20_std
value: 29.1586034119739
- type: nauc_mrr_at_3_diff1
value: 32.561539251580434
- type: nauc_mrr_at_3_max
value: 37.9193528344707
- type: nauc_mrr_at_3_std
value: 25.877386804749847
- type: nauc_mrr_at_5_diff1
value: 32.25306495173547
- type: nauc_mrr_at_5_max
value: 39.083851848069294
- type: nauc_mrr_at_5_std
value: 27.530721185638736
- type: nauc_ndcg_at_1000_diff1
value: 29.561218459059024
- type: nauc_ndcg_at_1000_max
value: 43.6835614353733
- type: nauc_ndcg_at_1000_std
value: 35.92798784055443
- type: nauc_ndcg_at_100_diff1
value: 29.94536984280927
- type: nauc_ndcg_at_100_max
value: 43.8410335236575
- type: nauc_ndcg_at_100_std
value: 35.505438612557036
- type: nauc_ndcg_at_10_diff1
value: 30.235348548493192
- type: nauc_ndcg_at_10_max
value: 41.82573840073112
- type: nauc_ndcg_at_10_std
value: 31.582058583570277
- type: nauc_ndcg_at_1_diff1
value: 37.75122698222711
- type: nauc_ndcg_at_1_max
value: 35.80113036602084
- type: nauc_ndcg_at_1_std
value: 22.20249486383813
- type: nauc_ndcg_at_20_diff1
value: 30.458353047565566
- type: nauc_ndcg_at_20_max
value: 43.01021607354784
- type: nauc_ndcg_at_20_std
value: 33.55606232519931
- type: nauc_ndcg_at_3_diff1
value: 30.860260937591256
- type: nauc_ndcg_at_3_max
value: 37.58324449207396
- type: nauc_ndcg_at_3_std
value: 25.835126314334556
- type: nauc_ndcg_at_5_diff1
value: 30.335040779381007
- type: nauc_ndcg_at_5_max
value: 39.89828354354357
- type: nauc_ndcg_at_5_std
value: 28.891396194531062
- type: nauc_precision_at_1000_diff1
value: 15.552294113688191
- type: nauc_precision_at_1000_max
value: 41.53406485867871
- type: nauc_precision_at_1000_std
value: 46.586530532497974
- type: nauc_precision_at_100_diff1
value: 21.999921653563852
- type: nauc_precision_at_100_max
value: 48.2700803515746
- type: nauc_precision_at_100_std
value: 48.57325173474392
- type: nauc_precision_at_10_diff1
value: 25.70658892601051
- type: nauc_precision_at_10_max
value: 46.94888455540377
- type: nauc_precision_at_10_std
value: 39.72188834633382
- type: nauc_precision_at_1_diff1
value: 37.75122698222711
- type: nauc_precision_at_1_max
value: 35.80113036602084
- type: nauc_precision_at_1_std
value: 22.20249486383813
- type: nauc_precision_at_20_diff1
value: 25.81577606956878
- type: nauc_precision_at_20_max
value: 48.78383366637263
- type: nauc_precision_at_20_std
value: 43.87045167647826
- type: nauc_precision_at_3_diff1
value: 27.531165103421557
- type: nauc_precision_at_3_max
value: 40.14696590677932
- type: nauc_precision_at_3_std
value: 29.899294615610124
- type: nauc_precision_at_5_diff1
value: 25.93502440358384
- type: nauc_precision_at_5_max
value: 44.00785804863087
- type: nauc_precision_at_5_std
value: 35.03277433042901
- type: nauc_recall_at_1000_diff1
value: 21.208987311784337
- type: nauc_recall_at_1000_max
value: 53.14694529463616
- type: nauc_recall_at_1000_std
value: 57.390851786916905
- type: nauc_recall_at_100_diff1
value: 24.66517693117814
- type: nauc_recall_at_100_max
value: 52.42144955311691
- type: nauc_recall_at_100_std
value: 51.618093583567926
- type: nauc_recall_at_10_diff1
value: 26.2614095401863
- type: nauc_recall_at_10_max
value: 46.16907420690962
- type: nauc_recall_at_10_std
value: 38.80812141228197
- type: nauc_recall_at_1_diff1
value: 37.771167547385545
- type: nauc_recall_at_1_max
value: 33.79936534835281
- type: nauc_recall_at_1_std
value: 19.8026545196434
- type: nauc_recall_at_20_diff1
value: 26.974114073935862
- type: nauc_recall_at_20_max
value: 49.22011806908929
- type: nauc_recall_at_20_std
value: 43.99043967687484
- type: nauc_recall_at_3_diff1
value: 27.362156493610396
- type: nauc_recall_at_3_max
value: 38.3365047861871
- type: nauc_recall_at_3_std
value: 27.661439277596077
- type: nauc_recall_at_5_diff1
value: 26.22976856531024
- type: nauc_recall_at_5_max
value: 41.809539450841896
- type: nauc_recall_at_5_std
value: 32.72369291405573
- type: ndcg_at_1
value: 12.803999999999998
- type: ndcg_at_10
value: 20.498
- type: ndcg_at_100
value: 24.5
- type: ndcg_at_1000
value: 26.47
- type: ndcg_at_20
value: 22.105
- type: ndcg_at_3
value: 16.399
- type: ndcg_at_5
value: 18.443
- type: precision_at_1
value: 12.803999999999998
- type: precision_at_10
value: 3.581
- type: precision_at_100
value: 0.581
- type: precision_at_1000
value: 0.077
- type: precision_at_20
value: 2.149
- type: precision_at_3
value: 7.609000000000001
- type: precision_at_5
value: 5.707
- type: recall_at_1
value: 11.006
- type: recall_at_10
value: 30.171
- type: recall_at_100
value: 48.778
- type: recall_at_1000
value: 64.21900000000001
- type: recall_at_20
value: 36.221
- type: recall_at_3
value: 19.322
- type: recall_at_5
value: 24.078
task:
type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval (default)
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
split: test
type: mteb/quora
metrics:
- type: main_score
value: 71.554
- type: map_at_1
value: 54.742999999999995
- type: map_at_10
value: 66.50500000000001
- type: map_at_100
value: 67.331
- type: map_at_1000
value: 67.378
- type: map_at_20
value: 67
- type: map_at_3
value: 63.641000000000005
- type: map_at_5
value: 65.33200000000001
- type: mrr_at_1
value: 62.72
- type: mrr_at_10
value: 70.89299206349173
- type: mrr_at_100
value: 71.27496005671617
- type: mrr_at_1000
value: 71.28806157709408
- type: mrr_at_20
value: 71.13000691890626
- type: mrr_at_3
value: 69.10333333333301
- type: mrr_at_5
value: 70.21583333333277
- type: nauc_map_at_1000_diff1
value: 66.65994150880708
- type: nauc_map_at_1000_max
value: 42.207657689146636
- type: nauc_map_at_1000_std
value: -4.614822708766726
- type: nauc_map_at_100_diff1
value: 66.65985460435601
- type: nauc_map_at_100_max
value: 42.1951418791399
- type: nauc_map_at_100_std
value: -4.668379257318548
- type: nauc_map_at_10_diff1
value: 66.65663432979525
- type: nauc_map_at_10_max
value: 41.728720685551366
- type: nauc_map_at_10_std
value: -5.48813041574439
- type: nauc_map_at_1_diff1
value: 68.58043728601021
- type: nauc_map_at_1_max
value: 35.94249158252442
- type: nauc_map_at_1_std
value: -7.845585809262184
- type: nauc_map_at_20_diff1
value: 66.61785931467136
- type: nauc_map_at_20_max
value: 42.01736888380615
- type: nauc_map_at_20_std
value: -5.027493425699964
- type: nauc_map_at_3_diff1
value: 66.98987257709052
- type: nauc_map_at_3_max
value: 40.35052681408878
- type: nauc_map_at_3_std
value: -7.050766461084552
- type: nauc_map_at_5_diff1
value: 66.75726460818723
- type: nauc_map_at_5_max
value: 41.098088880186914
- type: nauc_map_at_5_std
value: -6.347264380995132
- type: nauc_mrr_at_1000_diff1
value: 68.09159158010036
- type: nauc_mrr_at_1000_max
value: 44.78573778444799
- type: nauc_mrr_at_1000_std
value: -2.1984227944596064
- type: nauc_mrr_at_100_diff1
value: 68.08976216482941
- type: nauc_mrr_at_100_max
value: 44.79292331082449
- type: nauc_mrr_at_100_std
value: -2.192358511474152
- type: nauc_mrr_at_10_diff1
value: 67.97856869515971
- type: nauc_mrr_at_10_max
value: 44.75658442119892
- type: nauc_mrr_at_10_std
value: -2.266089652581688
- type: nauc_mrr_at_1_diff1
value: 70.47504526794133
- type: nauc_mrr_at_1_max
value: 42.998979288981566
- type: nauc_mrr_at_1_std
value: -3.844595032747518
- type: nauc_mrr_at_20_diff1
value: 68.04847259578324
- type: nauc_mrr_at_20_max
value: 44.77363717298844
- type: nauc_mrr_at_20_std
value: -2.220330904615823
- type: nauc_mrr_at_3_diff1
value: 68.15770162945928
- type: nauc_mrr_at_3_max
value: 44.63142381130085
- type: nauc_mrr_at_3_std
value: -2.8196051300720657
- type: nauc_mrr_at_5_diff1
value: 67.97716933879973
- type: nauc_mrr_at_5_max
value: 44.652057774354795
- type: nauc_mrr_at_5_std
value: -2.4832835704033376
- type: nauc_ndcg_at_1000_diff1
value: 66.3484597736734
- type: nauc_ndcg_at_1000_max
value: 44.274116813951935
- type: nauc_ndcg_at_1000_std
value: -1.5242443165970598
- type: nauc_ndcg_at_100_diff1
value: 66.29711869066402
- type: nauc_ndcg_at_100_max
value: 44.34137131282743
- type: nauc_ndcg_at_100_std
value: -1.7237366001412513
- type: nauc_ndcg_at_10_diff1
value: 65.77635604345431
- type: nauc_ndcg_at_10_max
value: 43.48141975314033
- type: nauc_ndcg_at_10_std
value: -3.4515731967276566
- type: nauc_ndcg_at_1_diff1
value: 70.45469179751821
- type: nauc_ndcg_at_1_max
value: 43.09778577908869
- type: nauc_ndcg_at_1_std
value: -3.7479094392549808
- type: nauc_ndcg_at_20_diff1
value: 65.93436529633378
- type: nauc_ndcg_at_20_max
value: 43.84308731244128
- type: nauc_ndcg_at_20_std
value: -2.8872447228365674
- type: nauc_ndcg_at_3_diff1
value: 66.27497432752165
- type: nauc_ndcg_at_3_max
value: 42.688109359124404
- type: nauc_ndcg_at_3_std
value: -4.5986707014568
- type: nauc_ndcg_at_5_diff1
value: 65.89551276242311
- type: nauc_ndcg_at_5_max
value: 42.8152724638242
- type: nauc_ndcg_at_5_std
value: -4.359932658329261
- type: nauc_precision_at_1000_diff1
value: -25.845771358270355
- type: nauc_precision_at_1000_max
value: 1.2507066004434095
- type: nauc_precision_at_1000_std
value: 26.783890797373207
- type: nauc_precision_at_100_diff1
value: -19.728378500279163
- type: nauc_precision_at_100_max
value: 6.959031535233483
- type: nauc_precision_at_100_std
value: 25.84773456899786
- type: nauc_precision_at_10_diff1
value: 0.16708404784906175
- type: nauc_precision_at_10_max
value: 19.543191924357625
- type: nauc_precision_at_10_std
value: 16.79289511069038
- type: nauc_precision_at_1_diff1
value: 70.45469179751821
- type: nauc_precision_at_1_max
value: 43.09778577908869
- type: nauc_precision_at_1_std
value: -3.7479094392549808
- type: nauc_precision_at_20_diff1
value: -8.382255500761856
- type: nauc_precision_at_20_max
value: 15.035473959289568
- type: nauc_precision_at_20_std
value: 20.538740885480326
- type: nauc_precision_at_3_diff1
value: 27.048709249986878
- type: nauc_precision_at_3_max
value: 30.518546736431258
- type: nauc_precision_at_3_std
value: 5.306908786841801
- type: nauc_precision_at_5_diff1
value: 14.25364007698366
- type: nauc_precision_at_5_max
value: 25.54441584174495
- type: nauc_precision_at_5_std
value: 10.355093942273076
- type: nauc_recall_at_1000_diff1
value: 52.45597231858965
- type: nauc_recall_at_1000_max
value: 60.33888868417243
- type: nauc_recall_at_1000_std
value: 41.74015618938176
- type: nauc_recall_at_100_diff1
value: 55.86982273031775
- type: nauc_recall_at_100_max
value: 51.926932990913464
- type: nauc_recall_at_100_std
value: 16.958268019392627
- type: nauc_recall_at_10_diff1
value: 57.73562962930997
- type: nauc_recall_at_10_max
value: 43.066024842226554
- type: nauc_recall_at_10_std
value: -2.132404757799147
- type: nauc_recall_at_1_diff1
value: 68.58043728601021
- type: nauc_recall_at_1_max
value: 35.94249158252442
- type: nauc_recall_at_1_std
value: -7.845585809262184
- type: nauc_recall_at_20_diff1
value: 57.290031504158435
- type: nauc_recall_at_20_max
value: 45.45886613788042
- type: nauc_recall_at_20_std
value: 1.6370090149939058
- type: nauc_recall_at_3_diff1
value: 62.08876517630152
- type: nauc_recall_at_3_max
value: 40.23002104011903
- type: nauc_recall_at_3_std
value: -6.869162989714192
- type: nauc_recall_at_5_diff1
value: 59.933018486917746
- type: nauc_recall_at_5_max
value: 40.88657267886578
- type: nauc_recall_at_5_std
value: -5.628050779443181
- type: ndcg_at_1
value: 62.73
- type: ndcg_at_10
value: 71.554
- type: ndcg_at_100
value: 74.311
- type: ndcg_at_1000
value: 75.019
- type: ndcg_at_20
value: 72.75500000000001
- type: ndcg_at_3
value: 67.54400000000001
- type: ndcg_at_5
value: 69.55
- type: precision_at_1
value: 62.73
- type: precision_at_10
value: 10.906
- type: precision_at_100
value: 1.35
- type: precision_at_1000
value: 0.149
- type: precision_at_20
value: 5.915
- type: precision_at_3
value: 29.263
- type: precision_at_5
value: 19.488
- type: recall_at_1
value: 54.742999999999995
- type: recall_at_10
value: 81.716
- type: recall_at_100
value: 92.851
- type: recall_at_1000
value: 97.273
- type: recall_at_20
value: 85.92
- type: recall_at_3
value: 70.498
- type: recall_at_5
value: 75.849
task:
type: Retrieval
- dataset:
config: default
name: MTEB SCIDOCS (default)
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
split: test
type: mteb/scidocs
metrics:
- type: main_score
value: 13.517999999999999
- type: map_at_1
value: 3.5229999999999997
- type: map_at_10
value: 7.915
- type: map_at_100
value: 8.997
- type: map_at_1000
value: 9.176
- type: map_at_20
value: 8.461
- type: map_at_3
value: 6.053
- type: map_at_5
value: 6.984
- type: mrr_at_1
value: 17.299999999999997
- type: mrr_at_10
value: 24.74190476190474
- type: mrr_at_100
value: 25.9101667039407
- type: mrr_at_1000
value: 25.988199023946144
- type: mrr_at_20
value: 25.481046473306517
- type: mrr_at_3
value: 22.216666666666676
- type: mrr_at_5
value: 23.56166666666666
- type: nauc_map_at_1000_diff1
value: 24.455910313503278
- type: nauc_map_at_1000_max
value: 33.83004737315066
- type: nauc_map_at_1000_std
value: 9.194523320442991
- type: nauc_map_at_100_diff1
value: 24.339630376832936
- type: nauc_map_at_100_max
value: 33.61356299791679
- type: nauc_map_at_100_std
value: 8.882372757049831
- type: nauc_map_at_10_diff1
value: 25.017917879428097
- type: nauc_map_at_10_max
value: 32.33504087012955
- type: nauc_map_at_10_std
value: 6.734267884868271
- type: nauc_map_at_1_diff1
value: 33.07304758382069
- type: nauc_map_at_1_max
value: 25.491260816416794
- type: nauc_map_at_1_std
value: 1.7848545220909784
- type: nauc_map_at_20_diff1
value: 24.198637656629334
- type: nauc_map_at_20_max
value: 32.690638369259
- type: nauc_map_at_20_std
value: 7.586937464360605
- type: nauc_map_at_3_diff1
value: 26.205857642960584
- type: nauc_map_at_3_max
value: 29.44632005065384
- type: nauc_map_at_3_std
value: 3.4668077481217945
- type: nauc_map_at_5_diff1
value: 25.34442003192351
- type: nauc_map_at_5_max
value: 31.284146209449215
- type: nauc_map_at_5_std
value: 4.52506460246591
- type: nauc_mrr_at_1000_diff1
value: 26.631593454876302
- type: nauc_mrr_at_1000_max
value: 26.390156952979044
- type: nauc_mrr_at_1000_std
value: 5.500122607349167
- type: nauc_mrr_at_100_diff1
value: 26.611211042144717
- type: nauc_mrr_at_100_max
value: 26.39311562918225
- type: nauc_mrr_at_100_std
value: 5.525157281121166
- type: nauc_mrr_at_10_diff1
value: 26.461097777147565
- type: nauc_mrr_at_10_max
value: 25.91630664617085
- type: nauc_mrr_at_10_std
value: 4.953513051323053
- type: nauc_mrr_at_1_diff1
value: 32.8673947296537
- type: nauc_mrr_at_1_max
value: 25.306851956706964
- type: nauc_mrr_at_1_std
value: 1.683276319621701
- type: nauc_mrr_at_20_diff1
value: 26.35674529364598
- type: nauc_mrr_at_20_max
value: 26.16462178058776
- type: nauc_mrr_at_20_std
value: 5.36435930561749
- type: nauc_mrr_at_3_diff1
value: 27.80047318173256
- type: nauc_mrr_at_3_max
value: 25.872566241336457
- type: nauc_mrr_at_3_std
value: 3.557578523875398
- type: nauc_mrr_at_5_diff1
value: 27.047404596217532
- type: nauc_mrr_at_5_max
value: 26.279644475539182
- type: nauc_mrr_at_5_std
value: 4.369918153935665
- type: nauc_ndcg_at_1000_diff1
value: 23.4456737578451
- type: nauc_ndcg_at_1000_max
value: 36.034352189412054
- type: nauc_ndcg_at_1000_std
value: 16.484156459545886
- type: nauc_ndcg_at_100_diff1
value: 22.82831843216624
- type: nauc_ndcg_at_100_max
value: 34.1365140147972
- type: nauc_ndcg_at_100_std
value: 14.483227399870668
- type: nauc_ndcg_at_10_diff1
value: 23.143636366201108
- type: nauc_ndcg_at_10_max
value: 30.284313035992454
- type: nauc_ndcg_at_10_std
value: 8.236248942465304
- type: nauc_ndcg_at_1_diff1
value: 32.8673947296537
- type: nauc_ndcg_at_1_max
value: 25.306851956706964
- type: nauc_ndcg_at_1_std
value: 1.683276319621701
- type: nauc_ndcg_at_20_diff1
value: 22.10270448324872
- type: nauc_ndcg_at_20_max
value: 31.087148948000117
- type: nauc_ndcg_at_20_std
value: 9.944104633885981
- type: nauc_ndcg_at_3_diff1
value: 25.440248950989997
- type: nauc_ndcg_at_3_max
value: 28.246623874276388
- type: nauc_ndcg_at_3_std
value: 4.130613893552461
- type: nauc_ndcg_at_5_diff1
value: 24.123293565265154
- type: nauc_ndcg_at_5_max
value: 29.79631494997637
- type: nauc_ndcg_at_5_std
value: 5.568162599811792
- type: nauc_precision_at_1000_diff1
value: 15.29944955481792
- type: nauc_precision_at_1000_max
value: 36.79591811647109
- type: nauc_precision_at_1000_std
value: 28.258684253207196
- type: nauc_precision_at_100_diff1
value: 15.474631055719923
- type: nauc_precision_at_100_max
value: 34.64308589262915
- type: nauc_precision_at_100_std
value: 24.74903552712396
- type: nauc_precision_at_10_diff1
value: 17.66164436305463
- type: nauc_precision_at_10_max
value: 30.27724054726259
- type: nauc_precision_at_10_std
value: 12.045790012185199
- type: nauc_precision_at_1_diff1
value: 32.8673947296537
- type: nauc_precision_at_1_max
value: 25.306851956706964
- type: nauc_precision_at_1_std
value: 1.683276319621701
- type: nauc_precision_at_20_diff1
value: 14.925311182923828
- type: nauc_precision_at_20_max
value: 30.3754730346363
- type: nauc_precision_at_20_std
value: 14.967037084167767
- type: nauc_precision_at_3_diff1
value: 22.12097232035023
- type: nauc_precision_at_3_max
value: 28.90145998417316
- type: nauc_precision_at_3_std
value: 5.207826293980497
- type: nauc_precision_at_5_diff1
value: 19.646124390015505
- type: nauc_precision_at_5_max
value: 30.700015502912127
- type: nauc_precision_at_5_std
value: 7.427940969317621
- type: nauc_recall_at_1000_diff1
value: 14.848430988160194
- type: nauc_recall_at_1000_max
value: 36.302276276172904
- type: nauc_recall_at_1000_std
value: 27.627870225767538
- type: nauc_recall_at_100_diff1
value: 14.927004116409107
- type: nauc_recall_at_100_max
value: 34.122912157682684
- type: nauc_recall_at_100_std
value: 24.23381166510311
- type: nauc_recall_at_10_diff1
value: 17.52209648033621
- type: nauc_recall_at_10_max
value: 29.945321845042532
- type: nauc_recall_at_10_std
value: 11.800345352046559
- type: nauc_recall_at_1_diff1
value: 33.07304758382069
- type: nauc_recall_at_1_max
value: 25.491260816416794
- type: nauc_recall_at_1_std
value: 1.7848545220909784
- type: nauc_recall_at_20_diff1
value: 14.653160966809933
- type: nauc_recall_at_20_max
value: 29.918437729961667
- type: nauc_recall_at_20_std
value: 14.617926536658679
- type: nauc_recall_at_3_diff1
value: 22.180882902500866
- type: nauc_recall_at_3_max
value: 28.955925628304303
- type: nauc_recall_at_3_std
value: 5.147959983230342
- type: nauc_recall_at_5_diff1
value: 19.606013374343526
- type: nauc_recall_at_5_max
value: 30.729967797498414
- type: nauc_recall_at_5_std
value: 7.420579815013824
- type: ndcg_at_1
value: 17.299999999999997
- type: ndcg_at_10
value: 13.517999999999999
- type: ndcg_at_100
value: 18.748
- type: ndcg_at_1000
value: 22.545
- type: ndcg_at_20
value: 15.382000000000001
- type: ndcg_at_3
value: 13.471
- type: ndcg_at_5
value: 11.451
- type: precision_at_1
value: 17.299999999999997
- type: precision_at_10
value: 6.7299999999999995
- type: precision_at_100
value: 1.43
- type: precision_at_1000
value: 0.234
- type: precision_at_20
value: 4.445
- type: precision_at_3
value: 12.367
- type: precision_at_5
value: 9.74
- type: recall_at_1
value: 3.5229999999999997
- type: recall_at_10
value: 13.683
- type: recall_at_100
value: 29.098000000000003
- type: recall_at_1000
value: 47.595
- type: recall_at_20
value: 18.073
- type: recall_at_3
value: 7.528
- type: recall_at_5
value: 9.905
task:
type: Retrieval
- dataset:
config: default
name: MTEB SciFact (default)
revision: 0228b52cf27578f30900b9e5271d331663a030d7
split: test
type: mteb/scifact
metrics:
- type: main_score
value: 67.45100000000001
- type: map_at_1
value: 53.65
- type: map_at_10
value: 62.989
- type: map_at_100
value: 63.56699999999999
- type: map_at_1000
value: 63.589
- type: map_at_20
value: 63.322
- type: map_at_3
value: 60.428000000000004
- type: map_at_5
value: 61.746
- type: mrr_at_1
value: 56.333333333333336
- type: mrr_at_10
value: 64.13531746031748
- type: mrr_at_100
value: 64.6502313045011
- type: mrr_at_1000
value: 64.67013730823842
- type: mrr_at_20
value: 64.43527267939034
- type: mrr_at_3
value: 62.16666666666668
- type: mrr_at_5
value: 63.11666666666667
- type: nauc_map_at_1000_diff1
value: 67.43871554709371
- type: nauc_map_at_1000_max
value: 56.87707400987863
- type: nauc_map_at_1000_std
value: 13.201817410562603
- type: nauc_map_at_100_diff1
value: 67.43014288014047
- type: nauc_map_at_100_max
value: 56.883927550891386
- type: nauc_map_at_100_std
value: 13.204268004519518
- type: nauc_map_at_10_diff1
value: 67.49162463152845
- type: nauc_map_at_10_max
value: 56.97644107687605
- type: nauc_map_at_10_std
value: 13.004767022411087
- type: nauc_map_at_1_diff1
value: 67.8603693433359
- type: nauc_map_at_1_max
value: 49.43251795202199
- type: nauc_map_at_1_std
value: 6.912136879424281
- type: nauc_map_at_20_diff1
value: 67.32806421479054
- type: nauc_map_at_20_max
value: 56.872490664449984
- type: nauc_map_at_20_std
value: 13.30541427931393
- type: nauc_map_at_3_diff1
value: 68.40309668384425
- type: nauc_map_at_3_max
value: 55.585348937448686
- type: nauc_map_at_3_std
value: 10.637842848724745
- type: nauc_map_at_5_diff1
value: 67.6085990023231
- type: nauc_map_at_5_max
value: 56.24576150346847
- type: nauc_map_at_5_std
value: 11.870434642467732
- type: nauc_mrr_at_1000_diff1
value: 67.62060043481964
- type: nauc_mrr_at_1000_max
value: 58.36047480075881
- type: nauc_mrr_at_1000_std
value: 15.204349716076566
- type: nauc_mrr_at_100_diff1
value: 67.61097799902636
- type: nauc_mrr_at_100_max
value: 58.36616990826231
- type: nauc_mrr_at_100_std
value: 15.205918705276003
- type: nauc_mrr_at_10_diff1
value: 67.64119974776985
- type: nauc_mrr_at_10_max
value: 58.53357486892739
- type: nauc_mrr_at_10_std
value: 15.183780488996309
- type: nauc_mrr_at_1_diff1
value: 68.04114409455924
- type: nauc_mrr_at_1_max
value: 53.395195106778026
- type: nauc_mrr_at_1_std
value: 11.22962146863143
- type: nauc_mrr_at_20_diff1
value: 67.51089523977562
- type: nauc_mrr_at_20_max
value: 58.39065359646342
- type: nauc_mrr_at_20_std
value: 15.397131327128541
- type: nauc_mrr_at_3_diff1
value: 68.66391310043399
- type: nauc_mrr_at_3_max
value: 58.72513527089311
- type: nauc_mrr_at_3_std
value: 14.482335621031536
- type: nauc_mrr_at_5_diff1
value: 67.63339880758966
- type: nauc_mrr_at_5_max
value: 58.49875096834214
- type: nauc_mrr_at_5_std
value: 15.18233662283586
- type: nauc_ndcg_at_1000_diff1
value: 67.36797665645172
- type: nauc_ndcg_at_1000_max
value: 58.99334315671615
- type: nauc_ndcg_at_1000_std
value: 15.650591475261757
- type: nauc_ndcg_at_100_diff1
value: 67.04578186513078
- type: nauc_ndcg_at_100_max
value: 59.12903176643754
- type: nauc_ndcg_at_100_std
value: 15.873975219475037
- type: nauc_ndcg_at_10_diff1
value: 66.84396286068795
- type: nauc_ndcg_at_10_max
value: 59.74397228268845
- type: nauc_ndcg_at_10_std
value: 15.888728500432956
- type: nauc_ndcg_at_1_diff1
value: 68.04114409455924
- type: nauc_ndcg_at_1_max
value: 53.395195106778026
- type: nauc_ndcg_at_1_std
value: 11.22962146863143
- type: nauc_ndcg_at_20_diff1
value: 66.25089652332008
- type: nauc_ndcg_at_20_max
value: 59.267825244207565
- type: nauc_ndcg_at_20_std
value: 16.8645668167251
- type: nauc_ndcg_at_3_diff1
value: 68.41523076040451
- type: nauc_ndcg_at_3_max
value: 58.416730090539424
- type: nauc_ndcg_at_3_std
value: 13.099839531734622
- type: nauc_ndcg_at_5_diff1
value: 67.03081040284573
- type: nauc_ndcg_at_5_max
value: 58.63120247252088
- type: nauc_ndcg_at_5_std
value: 14.118200679420816
- type: nauc_precision_at_1000_diff1
value: -13.47492055830411
- type: nauc_precision_at_1000_max
value: 17.212367429575043
- type: nauc_precision_at_1000_std
value: 35.200380783824734
- type: nauc_precision_at_100_diff1
value: 1.7976055965574322
- type: nauc_precision_at_100_max
value: 30.693920234632362
- type: nauc_precision_at_100_std
value: 37.524340697646494
- type: nauc_precision_at_10_diff1
value: 24.447664761758
- type: nauc_precision_at_10_max
value: 47.617098922049614
- type: nauc_precision_at_10_std
value: 34.29653987068447
- type: nauc_precision_at_1_diff1
value: 68.04114409455924
- type: nauc_precision_at_1_max
value: 53.395195106778026
- type: nauc_precision_at_1_std
value: 11.22962146863143
- type: nauc_precision_at_20_diff1
value: 14.803856365569235
- type: nauc_precision_at_20_max
value: 42.28753070062891
- type: nauc_precision_at_20_std
value: 40.424284001110095
- type: nauc_precision_at_3_diff1
value: 51.73629472942003
- type: nauc_precision_at_3_max
value: 56.08453509035213
- type: nauc_precision_at_3_std
value: 21.17712635956948
- type: nauc_precision_at_5_diff1
value: 39.537844338381646
- type: nauc_precision_at_5_max
value: 53.42459318275764
- type: nauc_precision_at_5_std
value: 27.39110827163739
- type: nauc_recall_at_1000_diff1
value: 83.6430507162216
- type: nauc_recall_at_1000_max
value: 82.64956389059707
- type: nauc_recall_at_1000_std
value: 32.898362597071454
- type: nauc_recall_at_100_diff1
value: 64.42577030812336
- type: nauc_recall_at_100_max
value: 69.51777933395574
- type: nauc_recall_at_100_std
value: 27.661453470277102
- type: nauc_recall_at_10_diff1
value: 62.70521806358795
- type: nauc_recall_at_10_max
value: 67.83398886849211
- type: nauc_recall_at_10_std
value: 23.762450199550837
- type: nauc_recall_at_1_diff1
value: 67.8603693433359
- type: nauc_recall_at_1_max
value: 49.43251795202199
- type: nauc_recall_at_1_std
value: 6.912136879424281
- type: nauc_recall_at_20_diff1
value: 58.03251739041354
- type: nauc_recall_at_20_max
value: 66.39099614909134
- type: nauc_recall_at_20_std
value: 31.180896538363452
- type: nauc_recall_at_3_diff1
value: 68.85151954947838
- type: nauc_recall_at_3_max
value: 62.448734183879715
- type: nauc_recall_at_3_std
value: 13.972637930457147
- type: nauc_recall_at_5_diff1
value: 64.05212433203268
- type: nauc_recall_at_5_max
value: 63.05508769069436
- type: nauc_recall_at_5_std
value: 17.335334626975175
- type: ndcg_at_1
value: 56.333
- type: ndcg_at_10
value: 67.45100000000001
- type: ndcg_at_100
value: 70.09100000000001
- type: ndcg_at_1000
value: 70.757
- type: ndcg_at_20
value: 68.577
- type: ndcg_at_3
value: 62.857
- type: ndcg_at_5
value: 64.72
- type: precision_at_1
value: 56.333
- type: precision_at_10
value: 8.933
- type: precision_at_100
value: 1.0330000000000001
- type: precision_at_1000
value: 0.11
- type: precision_at_20
value: 4.717
- type: precision_at_3
value: 24.333
- type: precision_at_5
value: 15.867
- type: recall_at_1
value: 53.65
- type: recall_at_10
value: 80.061
- type: recall_at_100
value: 92
- type: recall_at_1000
value: 97.26700000000001
- type: recall_at_20
value: 84.378
- type: recall_at_3
value: 67.428
- type: recall_at_5
value: 72.09400000000001
task:
type: Retrieval
- dataset:
config: default
name: MTEB T2Retrieval (default)
revision: 8731a845f1bf500a4f111cf1070785c793d10e64
split: test
type: C-MTEB/T2Retrieval
metrics:
- type: main_score
value: 70.56400000000001
- type: map_at_1
value: 23.343
- type: map_at_10
value: 61.009
- type: map_at_100
value: 64.43100000000001
- type: map_at_1000
value: 64.562
- type: map_at_20
value: 63.395999999999994
- type: map_at_3
value: 44.344
- type: map_at_5
value: 53.504
- type: mrr_at_1
value: 80.19463440294582
- type: mrr_at_10
value: 84.12514368655899
- type: mrr_at_100
value: 84.31811888616042
- type: mrr_at_1000
value: 84.32771155206774
- type: mrr_at_20
value: 84.25141792188293
- type: mrr_at_3
value: 83.37497808171138
- type: mrr_at_5
value: 83.83723478870802
- type: nauc_map_at_1000_diff1
value: 22.496957132886223
- type: nauc_map_at_1000_max
value: 55.96470814075969
- type: nauc_map_at_1000_std
value: 32.601895486818826
- type: nauc_map_at_100_diff1
value: 22.48888923975606
- type: nauc_map_at_100_max
value: 55.897135867165325
- type: nauc_map_at_100_std
value: 32.48193019235303
- type: nauc_map_at_10_diff1
value: 23.70249242804442
- type: nauc_map_at_10_max
value: 51.40247885329748
- type: nauc_map_at_10_std
value: 24.27336153842712
- type: nauc_map_at_1_diff1
value: 45.510735110050405
- type: nauc_map_at_1_max
value: 6.855375805557362
- type: nauc_map_at_1_std
value: -19.686156263829435
- type: nauc_map_at_20_diff1
value: 22.588880791842648
- type: nauc_map_at_20_max
value: 54.876016264954565
- type: nauc_map_at_20_std
value: 30.3867274764393
- type: nauc_map_at_3_diff1
value: 33.87565317249748
- type: nauc_map_at_3_max
value: 25.67974066213008
- type: nauc_map_at_3_std
value: -6.37135891416
- type: nauc_map_at_5_diff1
value: 28.62311641700491
- type: nauc_map_at_5_max
value: 38.30358147974758
- type: nauc_map_at_5_std
value: 6.890948784359749
- type: nauc_mrr_at_1000_diff1
value: 54.039101613763606
- type: nauc_mrr_at_1000_max
value: 76.45873864072455
- type: nauc_mrr_at_1000_std
value: 47.954681220242925
- type: nauc_mrr_at_100_diff1
value: 54.0402318626641
- type: nauc_mrr_at_100_max
value: 76.4644118238503
- type: nauc_mrr_at_100_std
value: 47.96421077433231
- type: nauc_mrr_at_10_diff1
value: 54.00311952299518
- type: nauc_mrr_at_10_max
value: 76.4883471662642
- type: nauc_mrr_at_10_std
value: 47.97540395073088
- type: nauc_mrr_at_1_diff1
value: 54.48089128029922
- type: nauc_mrr_at_1_max
value: 73.86293219090854
- type: nauc_mrr_at_1_std
value: 43.893078207399725
- type: nauc_mrr_at_20_diff1
value: 54.02405038081828
- type: nauc_mrr_at_20_max
value: 76.47926942691595
- type: nauc_mrr_at_20_std
value: 47.989225573091915
- type: nauc_mrr_at_3_diff1
value: 54.00341911971097
- type: nauc_mrr_at_3_max
value: 76.45961735371569
- type: nauc_mrr_at_3_std
value: 47.61426942204108
- type: nauc_mrr_at_5_diff1
value: 53.9619722001384
- type: nauc_mrr_at_5_max
value: 76.46317048080803
- type: nauc_mrr_at_5_std
value: 47.73472005225089
- type: nauc_ndcg_at_1000_diff1
value: 27.575467236770006
- type: nauc_ndcg_at_1000_max
value: 62.97783076405462
- type: nauc_ndcg_at_1000_std
value: 42.02022955089562
- type: nauc_ndcg_at_100_diff1
value: 27.15156601573711
- type: nauc_ndcg_at_100_max
value: 62.404979147635096
- type: nauc_ndcg_at_100_std
value: 41.2772459964177
- type: nauc_ndcg_at_10_diff1
value: 26.341363740340835
- type: nauc_ndcg_at_10_max
value: 59.09789857330006
- type: nauc_ndcg_at_10_std
value: 34.63284738139455
- type: nauc_ndcg_at_1_diff1
value: 54.48089128029922
- type: nauc_ndcg_at_1_max
value: 73.86293219090854
- type: nauc_ndcg_at_1_std
value: 43.893078207399725
- type: nauc_ndcg_at_20_diff1
value: 26.745690568011582
- type: nauc_ndcg_at_20_max
value: 60.47418608855715
- type: nauc_ndcg_at_20_std
value: 37.07806439202376
- type: nauc_ndcg_at_3_diff1
value: 27.448474158468805
- type: nauc_ndcg_at_3_max
value: 66.35144888436369
- type: nauc_ndcg_at_3_std
value: 43.0380315268214
- type: nauc_ndcg_at_5_diff1
value: 25.304764889456273
- type: nauc_ndcg_at_5_max
value: 62.58655126382475
- type: nauc_ndcg_at_5_std
value: 39.829814054023934
- type: nauc_precision_at_1000_diff1
value: -16.90157504432568
- type: nauc_precision_at_1000_max
value: 31.127590699930458
- type: nauc_precision_at_1000_std
value: 53.37248210552581
- type: nauc_precision_at_100_diff1
value: -15.935010943985944
- type: nauc_precision_at_100_max
value: 35.932780668252754
- type: nauc_precision_at_100_std
value: 56.314934387677475
- type: nauc_precision_at_10_diff1
value: -13.075680729640176
- type: nauc_precision_at_10_max
value: 46.28713840678731
- type: nauc_precision_at_10_std
value: 55.524572031300664
- type: nauc_precision_at_1_diff1
value: 54.48089128029922
- type: nauc_precision_at_1_max
value: 73.86293219090854
- type: nauc_precision_at_1_std
value: 43.893078207399725
- type: nauc_precision_at_20_diff1
value: -14.671291238209152
- type: nauc_precision_at_20_max
value: 42.18289329500435
- type: nauc_precision_at_20_std
value: 57.14838477872818
- type: nauc_precision_at_3_diff1
value: 0.4858379814922798
- type: nauc_precision_at_3_max
value: 59.22453366703292
- type: nauc_precision_at_3_std
value: 50.16608892838139
- type: nauc_precision_at_5_diff1
value: -8.296394992313592
- type: nauc_precision_at_5_max
value: 52.31908226512005
- type: nauc_precision_at_5_std
value: 52.27306483377752
- type: nauc_recall_at_1000_diff1
value: 14.665787859575602
- type: nauc_recall_at_1000_max
value: 48.0799159049133
- type: nauc_recall_at_1000_std
value: 45.084469451871485
- type: nauc_recall_at_100_diff1
value: 17.08454419888426
- type: nauc_recall_at_100_max
value: 51.26393807559256
- type: nauc_recall_at_100_std
value: 41.92224510594129
- type: nauc_recall_at_10_diff1
value: 20.483580918254
- type: nauc_recall_at_10_max
value: 46.13586062002455
- type: nauc_recall_at_10_std
value: 21.789034009353863
- type: nauc_recall_at_1_diff1
value: 45.510735110050405
- type: nauc_recall_at_1_max
value: 6.855375805557362
- type: nauc_recall_at_1_std
value: -19.686156263829435
- type: nauc_recall_at_20_diff1
value: 18.651897353965456
- type: nauc_recall_at_20_max
value: 50.41164976390756
- type: nauc_recall_at_20_std
value: 31.935251832807097
- type: nauc_recall_at_3_diff1
value: 31.300457278276166
- type: nauc_recall_at_3_max
value: 21.964520248331244
- type: nauc_recall_at_3_std
value: -8.56293395643492
- type: nauc_recall_at_5_diff1
value: 25.934453669962682
- type: nauc_recall_at_5_max
value: 32.93948421343696
- type: nauc_recall_at_5_std
value: 3.150406655915041
- type: ndcg_at_1
value: 80.19500000000001
- type: ndcg_at_10
value: 70.56400000000001
- type: ndcg_at_100
value: 75.248
- type: ndcg_at_1000
value: 76.712
- type: ndcg_at_20
value: 72.59
- type: ndcg_at_3
value: 73.894
- type: ndcg_at_5
value: 71.385
- type: precision_at_1
value: 80.19500000000001
- type: precision_at_10
value: 34.565
- type: precision_at_100
value: 4.41
- type: precision_at_1000
value: 0.47800000000000004
- type: precision_at_20
value: 19.553
- type: precision_at_3
value: 64.371
- type: precision_at_5
value: 52.748
- type: recall_at_1
value: 23.343
- type: recall_at_10
value: 68.598
- type: recall_at_100
value: 83.466
- type: recall_at_1000
value: 90.72
- type: recall_at_20
value: 75.077
- type: recall_at_3
value: 46.317
- type: recall_at_5
value: 57.489999999999995
task:
type: Retrieval
- dataset:
config: default
name: MTEB TRECCOVID (default)
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
split: test
type: mteb/trec-covid
metrics:
- type: main_score
value: 62.425
- type: map_at_1
value: 0.203
- type: map_at_10
value: 1.459
- type: map_at_100
value: 6.9430000000000005
- type: map_at_1000
value: 16.881
- type: map_at_20
value: 2.536
- type: map_at_3
value: 0.5309999999999999
- type: map_at_5
value: 0.8210000000000001
- type: mrr_at_1
value: 76
- type: mrr_at_10
value: 83.65
- type: mrr_at_100
value: 83.72142857142858
- type: mrr_at_1000
value: 83.72142857142858
- type: mrr_at_20
value: 83.65
- type: mrr_at_3
value: 82
- type: mrr_at_5
value: 83.4
- type: nauc_map_at_1000_diff1
value: -6.541046681978992
- type: nauc_map_at_1000_max
value: 53.15454917103454
- type: nauc_map_at_1000_std
value: 79.85086181072259
- type: nauc_map_at_100_diff1
value: 2.9454404034728423
- type: nauc_map_at_100_max
value: 35.75982032790531
- type: nauc_map_at_100_std
value: 52.175513363150195
- type: nauc_map_at_10_diff1
value: 5.146390232256089
- type: nauc_map_at_10_max
value: 18.151667914188497
- type: nauc_map_at_10_std
value: 11.749174207425918
- type: nauc_map_at_1_diff1
value: 15.120130387703712
- type: nauc_map_at_1_max
value: -8.089044124963296
- type: nauc_map_at_1_std
value: -16.47698742886207
- type: nauc_map_at_20_diff1
value: 7.2409242766040816
- type: nauc_map_at_20_max
value: 21.74763641630688
- type: nauc_map_at_20_std
value: 17.84266990234575
- type: nauc_map_at_3_diff1
value: 6.559670685832701
- type: nauc_map_at_3_max
value: 2.4060769560268054
- type: nauc_map_at_3_std
value: -4.205911499342936
- type: nauc_map_at_5_diff1
value: 3.5437448801309843
- type: nauc_map_at_5_max
value: 8.507033857489805
- type: nauc_map_at_5_std
value: 2.557266246291616
- type: nauc_mrr_at_1000_diff1
value: 7.093561985479526
- type: nauc_mrr_at_1000_max
value: 12.163150377984188
- type: nauc_mrr_at_1000_std
value: 5.153490610556136
- type: nauc_mrr_at_100_diff1
value: 7.093561985479526
- type: nauc_mrr_at_100_max
value: 12.163150377984188
- type: nauc_mrr_at_100_std
value: 5.153490610556136
- type: nauc_mrr_at_10_diff1
value: 7.118234287248623
- type: nauc_mrr_at_10_max
value: 12.581138788928707
- type: nauc_mrr_at_10_std
value: 5.6048358251663695
- type: nauc_mrr_at_1_diff1
value: 8.224452712641664
- type: nauc_mrr_at_1_max
value: 4.5859652158864845
- type: nauc_mrr_at_1_std
value: -1.1205330102968079
- type: nauc_mrr_at_20_diff1
value: 7.118234287248623
- type: nauc_mrr_at_20_max
value: 12.581138788928707
- type: nauc_mrr_at_20_std
value: 5.6048358251663695
- type: nauc_mrr_at_3_diff1
value: 4.041734386967824
- type: nauc_mrr_at_3_max
value: 10.777335649834702
- type: nauc_mrr_at_3_std
value: 5.783780754357451
- type: nauc_mrr_at_5_diff1
value: 8.425724238641777
- type: nauc_mrr_at_5_max
value: 13.558212388947371
- type: nauc_mrr_at_5_std
value: 5.232323355931403
- type: nauc_ndcg_at_1000_diff1
value: -0.2949566425006407
- type: nauc_ndcg_at_1000_max
value: 52.40963371099916
- type: nauc_ndcg_at_1000_std
value: 77.11049722148121
- type: nauc_ndcg_at_100_diff1
value: -4.0697588078008256
- type: nauc_ndcg_at_100_max
value: 51.380824162720394
- type: nauc_ndcg_at_100_std
value: 71.07160892882715
- type: nauc_ndcg_at_10_diff1
value: -6.7144464670472
- type: nauc_ndcg_at_10_max
value: 39.47160981936148
- type: nauc_ndcg_at_10_std
value: 40.74701208709495
- type: nauc_ndcg_at_1_diff1
value: 11.078453972644848
- type: nauc_ndcg_at_1_max
value: 6.058859689565119
- type: nauc_ndcg_at_1_std
value: -0.5071461502996903
- type: nauc_ndcg_at_20_diff1
value: -3.887237913199889
- type: nauc_ndcg_at_20_max
value: 40.88921008584308
- type: nauc_ndcg_at_20_std
value: 46.15143090870543
- type: nauc_ndcg_at_3_diff1
value: -4.432884126905009
- type: nauc_ndcg_at_3_max
value: 22.35495334699474
- type: nauc_ndcg_at_3_std
value: 18.826073139000187
- type: nauc_ndcg_at_5_diff1
value: -8.409971354760488
- type: nauc_ndcg_at_5_max
value: 32.41394647563698
- type: nauc_ndcg_at_5_std
value: 30.9427692315429
- type: nauc_precision_at_1000_diff1
value: -12.968656488046928
- type: nauc_precision_at_1000_max
value: 51.48841492821185
- type: nauc_precision_at_1000_std
value: 71.5784478734716
- type: nauc_precision_at_100_diff1
value: -8.842878579029152
- type: nauc_precision_at_100_max
value: 52.255502868428664
- type: nauc_precision_at_100_std
value: 76.5474049004033
- type: nauc_precision_at_10_diff1
value: -15.628865979381384
- type: nauc_precision_at_10_max
value: 46.87671821305843
- type: nauc_precision_at_10_std
value: 52.50472508591065
- type: nauc_precision_at_1_diff1
value: 8.224452712641664
- type: nauc_precision_at_1_max
value: 4.5859652158864845
- type: nauc_precision_at_1_std
value: -1.1205330102968079
- type: nauc_precision_at_20_diff1
value: -8.97641282121417
- type: nauc_precision_at_20_max
value: 45.19920811680273
- type: nauc_precision_at_20_std
value: 53.300381614764426
- type: nauc_precision_at_3_diff1
value: -12.11792171127635
- type: nauc_precision_at_3_max
value: 28.68369618328903
- type: nauc_precision_at_3_std
value: 32.4284706010099
- type: nauc_precision_at_5_diff1
value: -16.405745971858067
- type: nauc_precision_at_5_max
value: 38.79919314664645
- type: nauc_precision_at_5_std
value: 42.92247511249398
- type: nauc_recall_at_1000_diff1
value: -3.3845770983196557
- type: nauc_recall_at_1000_max
value: 48.14582988919492
- type: nauc_recall_at_1000_std
value: 75.41433415832614
- type: nauc_recall_at_100_diff1
value: 8.022453736855866
- type: nauc_recall_at_100_max
value: 24.550775222784488
- type: nauc_recall_at_100_std
value: 41.23574137418627
- type: nauc_recall_at_10_diff1
value: 5.930661939423501
- type: nauc_recall_at_10_max
value: 14.758382148924712
- type: nauc_recall_at_10_std
value: 7.083457328035323
- type: nauc_recall_at_1_diff1
value: 15.120130387703712
- type: nauc_recall_at_1_max
value: -8.089044124963296
- type: nauc_recall_at_1_std
value: -16.47698742886207
- type: nauc_recall_at_20_diff1
value: 7.4333001198631194
- type: nauc_recall_at_20_max
value: 14.390383453037828
- type: nauc_recall_at_20_std
value: 10.826932690629755
- type: nauc_recall_at_3_diff1
value: 6.1181369477439445
- type: nauc_recall_at_3_max
value: 1.4029324116194293
- type: nauc_recall_at_3_std
value: -5.123246588187333
- type: nauc_recall_at_5_diff1
value: 3.3614024885818328
- type: nauc_recall_at_5_max
value: 7.443114255052484
- type: nauc_recall_at_5_std
value: 0.1719114991000565
- type: ndcg_at_1
value: 72
- type: ndcg_at_10
value: 62.425
- type: ndcg_at_100
value: 42.833
- type: ndcg_at_1000
value: 38.538
- type: ndcg_at_20
value: 57.818000000000005
- type: ndcg_at_3
value: 66.469
- type: ndcg_at_5
value: 64.878
- type: precision_at_1
value: 76
- type: precision_at_10
value: 67.4
- type: precision_at_100
value: 44.18
- type: precision_at_1000
value: 17.95
- type: precision_at_20
value: 61.8
- type: precision_at_3
value: 71.333
- type: precision_at_5
value: 69.6
- type: recall_at_1
value: 0.203
- type: recall_at_10
value: 1.688
- type: recall_at_100
value: 10.302
- type: recall_at_1000
value: 36.3
- type: recall_at_20
value: 3.117
- type: recall_at_3
value: 0.559
- type: recall_at_5
value: 0.897
task:
type: Retrieval
- dataset:
config: default
name: MTEB Touche2020 (default)
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
split: test
type: mteb/touche2020
metrics:
- type: main_score
value: 23.639
- type: map_at_1
value: 2.092
- type: map_at_10
value: 9.293
- type: map_at_100
value: 16.176
- type: map_at_1000
value: 17.538
- type: map_at_20
value: 12.368
- type: map_at_3
value: 5.018000000000001
- type: map_at_5
value: 6.253
- type: mrr_at_1
value: 28.57142857142857
- type: mrr_at_10
value: 43.21347586653709
- type: mrr_at_100
value: 44.567870149502795
- type: mrr_at_1000
value: 44.567870149502795
- type: mrr_at_20
value: 44.26892835056099
- type: mrr_at_3
value: 39.45578231292517
- type: mrr_at_5
value: 40.98639455782313
- type: nauc_map_at_1000_diff1
value: 8.113808505002755
- type: nauc_map_at_1000_max
value: -12.821141916586242
- type: nauc_map_at_1000_std
value: -18.146754733521558
- type: nauc_map_at_100_diff1
value: 7.501096016012217
- type: nauc_map_at_100_max
value: -14.489710951391796
- type: nauc_map_at_100_std
value: -20.992487558293142
- type: nauc_map_at_10_diff1
value: 13.301089748633393
- type: nauc_map_at_10_max
value: -13.271669557688393
- type: nauc_map_at_10_std
value: -30.280706768603714
- type: nauc_map_at_1_diff1
value: 18.11449551683917
- type: nauc_map_at_1_max
value: -22.066396220697545
- type: nauc_map_at_1_std
value: -34.40518059825958
- type: nauc_map_at_20_diff1
value: 5.598236453598427
- type: nauc_map_at_20_max
value: -18.513590591712813
- type: nauc_map_at_20_std
value: -29.669717323308603
- type: nauc_map_at_3_diff1
value: 9.778805992604187
- type: nauc_map_at_3_max
value: -19.74166856327524
- type: nauc_map_at_3_std
value: -31.311603338046833
- type: nauc_map_at_5_diff1
value: 15.7202727474612
- type: nauc_map_at_5_max
value: -17.356144865319813
- type: nauc_map_at_5_std
value: -32.8447208930918
- type: nauc_mrr_at_1000_diff1
value: 13.274692366618835
- type: nauc_mrr_at_1000_max
value: -13.366257875792126
- type: nauc_mrr_at_1000_std
value: -19.38850049312152
- type: nauc_mrr_at_100_diff1
value: 13.274692366618835
- type: nauc_mrr_at_100_max
value: -13.366257875792126
- type: nauc_mrr_at_100_std
value: -19.38850049312152
- type: nauc_mrr_at_10_diff1
value: 14.895391726179636
- type: nauc_mrr_at_10_max
value: -12.021502471656444
- type: nauc_mrr_at_10_std
value: -18.515259456430417
- type: nauc_mrr_at_1_diff1
value: 12.265279179422803
- type: nauc_mrr_at_1_max
value: -20.588212783319833
- type: nauc_mrr_at_1_std
value: -27.417475766164507
- type: nauc_mrr_at_20_diff1
value: 13.238737984558973
- type: nauc_mrr_at_20_max
value: -13.560960724665147
- type: nauc_mrr_at_20_std
value: -19.492795685952842
- type: nauc_mrr_at_3_diff1
value: 12.671377368168827
- type: nauc_mrr_at_3_max
value: -11.864061527886694
- type: nauc_mrr_at_3_std
value: -19.3958302159125
- type: nauc_mrr_at_5_diff1
value: 12.643093703826533
- type: nauc_mrr_at_5_max
value: -12.019493556879212
- type: nauc_mrr_at_5_std
value: -18.49194959830366
- type: nauc_ndcg_at_1000_diff1
value: 12.96868651758221
- type: nauc_ndcg_at_1000_max
value: -2.4259137880152215
- type: nauc_ndcg_at_1000_std
value: 1.6992600264519455
- type: nauc_ndcg_at_100_diff1
value: 11.763108165833458
- type: nauc_ndcg_at_100_max
value: -7.955139386228701
- type: nauc_ndcg_at_100_std
value: -6.962889778793918
- type: nauc_ndcg_at_10_diff1
value: 9.313754328330232
- type: nauc_ndcg_at_10_max
value: -9.379074508658757
- type: nauc_ndcg_at_10_std
value: -21.289583949829353
- type: nauc_ndcg_at_1_diff1
value: 11.958955148338088
- type: nauc_ndcg_at_1_max
value: -21.45683911916427
- type: nauc_ndcg_at_1_std
value: -29.59828322405147
- type: nauc_ndcg_at_20_diff1
value: 5.702530833380917
- type: nauc_ndcg_at_20_max
value: -18.11189163555712
- type: nauc_ndcg_at_20_std
value: -24.899197257439077
- type: nauc_ndcg_at_3_diff1
value: 5.353711866477964
- type: nauc_ndcg_at_3_max
value: -13.383348388728653
- type: nauc_ndcg_at_3_std
value: -19.686710045051754
- type: nauc_ndcg_at_5_diff1
value: 6.850641359860924
- type: nauc_ndcg_at_5_max
value: -14.163098261933706
- type: nauc_ndcg_at_5_std
value: -21.86747182230662
- type: nauc_precision_at_1000_diff1
value: 0.36613794946631767
- type: nauc_precision_at_1000_max
value: 35.45296752706168
- type: nauc_precision_at_1000_std
value: 56.05545404548104
- type: nauc_precision_at_100_diff1
value: 7.077587391958853
- type: nauc_precision_at_100_max
value: 17.919073695918872
- type: nauc_precision_at_100_std
value: 41.866329607335764
- type: nauc_precision_at_10_diff1
value: 5.766871060833031
- type: nauc_precision_at_10_max
value: -3.5266859273978266
- type: nauc_precision_at_10_std
value: -13.922636784713669
- type: nauc_precision_at_1_diff1
value: 12.265279179422803
- type: nauc_precision_at_1_max
value: -20.588212783319833
- type: nauc_precision_at_1_std
value: -27.417475766164507
- type: nauc_precision_at_20_diff1
value: -4.721011610542265
- type: nauc_precision_at_20_max
value: -12.20792850759493
- type: nauc_precision_at_20_std
value: -6.800730490273968
- type: nauc_precision_at_3_diff1
value: 5.998524806193237
- type: nauc_precision_at_3_max
value: -12.521185750516828
- type: nauc_precision_at_3_std
value: -19.713926078179675
- type: nauc_precision_at_5_diff1
value: 7.969414594605033
- type: nauc_precision_at_5_max
value: -11.929523474817804
- type: nauc_precision_at_5_std
value: -22.238556874885905
- type: nauc_recall_at_1000_diff1
value: 25.629882948498878
- type: nauc_recall_at_1000_max
value: 14.847918788629963
- type: nauc_recall_at_1000_std
value: 36.0878682647753
- type: nauc_recall_at_100_diff1
value: 15.624671587346189
- type: nauc_recall_at_100_max
value: -5.489430694039881
- type: nauc_recall_at_100_std
value: 3.584655586077962
- type: nauc_recall_at_10_diff1
value: 19.377892005554312
- type: nauc_recall_at_10_max
value: -10.112413591039283
- type: nauc_recall_at_10_std
value: -28.87608804046421
- type: nauc_recall_at_1_diff1
value: 18.11449551683917
- type: nauc_recall_at_1_max
value: -22.066396220697545
- type: nauc_recall_at_1_std
value: -34.40518059825958
- type: nauc_recall_at_20_diff1
value: 4.044014829974238
- type: nauc_recall_at_20_max
value: -23.533354086997154
- type: nauc_recall_at_20_std
value: -29.349266698210375
- type: nauc_recall_at_3_diff1
value: 15.45704435080478
- type: nauc_recall_at_3_max
value: -15.27240567013374
- type: nauc_recall_at_3_std
value: -28.718036174306903
- type: nauc_recall_at_5_diff1
value: 22.26476137963053
- type: nauc_recall_at_5_max
value: -11.771781387076063
- type: nauc_recall_at_5_std
value: -29.986402693711156
- type: ndcg_at_1
value: 27.551
- type: ndcg_at_10
value: 23.639
- type: ndcg_at_100
value: 37.738
- type: ndcg_at_1000
value: 47.544
- type: ndcg_at_20
value: 26.579000000000004
- type: ndcg_at_3
value: 27.302
- type: ndcg_at_5
value: 23.643
- type: precision_at_1
value: 28.571
- type: precision_at_10
value: 21.429000000000002
- type: precision_at_100
value: 8.388
- type: precision_at_1000
value: 1.496
- type: precision_at_20
value: 19.184
- type: precision_at_3
value: 28.571
- type: precision_at_5
value: 23.265
- type: recall_at_1
value: 2.092
- type: recall_at_10
value: 15.1
- type: recall_at_100
value: 49.71
- type: recall_at_1000
value: 80.977
- type: recall_at_20
value: 25.372
- type: recall_at_3
value: 6.085999999999999
- type: recall_at_5
value: 8.265
task:
type: Retrieval
- dataset:
config: default
name: MTEB VideoRetrieval (default)
revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
split: test
type: C-MTEB/VideoRetrieval
metrics:
- type: main_score
value: 64.098
- type: map_at_1
value: 49.5
- type: map_at_10
value: 59.569
- type: map_at_100
value: 60.221999999999994
- type: map_at_1000
value: 60.242
- type: map_at_20
value: 60.01800000000001
- type: map_at_3
value: 57.416999999999994
- type: map_at_5
value: 58.916999999999994
- type: mrr_at_1
value: 49.5
- type: mrr_at_10
value: 59.56932539682539
- type: mrr_at_100
value: 60.222387235695905
- type: mrr_at_1000
value: 60.24159380617551
- type: mrr_at_20
value: 60.017881284951656
- type: mrr_at_3
value: 57.41666666666665
- type: mrr_at_5
value: 58.91666666666665
- type: nauc_map_at_1000_diff1
value: 70.5786375919516
- type: nauc_map_at_1000_max
value: 37.13543458080404
- type: nauc_map_at_1000_std
value: -17.600072228047342
- type: nauc_map_at_100_diff1
value: 70.56835550403025
- type: nauc_map_at_100_max
value: 37.1458814901935
- type: nauc_map_at_100_std
value: -17.6260192275505
- type: nauc_map_at_10_diff1
value: 70.41330031614633
- type: nauc_map_at_10_max
value: 36.97943280538239
- type: nauc_map_at_10_std
value: -18.04582458073953
- type: nauc_map_at_1_diff1
value: 73.69020304197174
- type: nauc_map_at_1_max
value: 30.80837706084608
- type: nauc_map_at_1_std
value: -18.555229677513733
- type: nauc_map_at_20_diff1
value: 70.50672518441239
- type: nauc_map_at_20_max
value: 37.155754338746945
- type: nauc_map_at_20_std
value: -17.872491967287978
- type: nauc_map_at_3_diff1
value: 70.9468365580479
- type: nauc_map_at_3_max
value: 37.076873199287995
- type: nauc_map_at_3_std
value: -17.8641399095897
- type: nauc_map_at_5_diff1
value: 70.75754048180886
- type: nauc_map_at_5_max
value: 37.07255888922384
- type: nauc_map_at_5_std
value: -18.087074198467132
- type: nauc_mrr_at_1000_diff1
value: 70.5786375919516
- type: nauc_mrr_at_1000_max
value: 37.13543458080404
- type: nauc_mrr_at_1000_std
value: -17.600072228047342
- type: nauc_mrr_at_100_diff1
value: 70.56835550403025
- type: nauc_mrr_at_100_max
value: 37.1458814901935
- type: nauc_mrr_at_100_std
value: -17.6260192275505
- type: nauc_mrr_at_10_diff1
value: 70.41330031614633
- type: nauc_mrr_at_10_max
value: 36.97943280538239
- type: nauc_mrr_at_10_std
value: -18.04582458073953
- type: nauc_mrr_at_1_diff1
value: 73.69020304197174
- type: nauc_mrr_at_1_max
value: 30.80837706084608
- type: nauc_mrr_at_1_std
value: -18.555229677513733
- type: nauc_mrr_at_20_diff1
value: 70.50672518441239
- type: nauc_mrr_at_20_max
value: 37.155754338746945
- type: nauc_mrr_at_20_std
value: -17.872491967287978
- type: nauc_mrr_at_3_diff1
value: 70.9468365580479
- type: nauc_mrr_at_3_max
value: 37.076873199287995
- type: nauc_mrr_at_3_std
value: -17.8641399095897
- type: nauc_mrr_at_5_diff1
value: 70.75754048180886
- type: nauc_mrr_at_5_max
value: 37.07255888922384
- type: nauc_mrr_at_5_std
value: -18.087074198467132
- type: nauc_ndcg_at_1000_diff1
value: 69.74559484105404
- type: nauc_ndcg_at_1000_max
value: 40.02888757631894
- type: nauc_ndcg_at_1000_std
value: -14.316316953552445
- type: nauc_ndcg_at_100_diff1
value: 69.51463571115605
- type: nauc_ndcg_at_100_max
value: 40.41444239756333
- type: nauc_ndcg_at_100_std
value: -14.698640780124517
- type: nauc_ndcg_at_10_diff1
value: 68.78624170599853
- type: nauc_ndcg_at_10_max
value: 39.383652814406304
- type: nauc_ndcg_at_10_std
value: -17.618038649579166
- type: nauc_ndcg_at_1_diff1
value: 73.69020304197174
- type: nauc_ndcg_at_1_max
value: 30.80837706084608
- type: nauc_ndcg_at_1_std
value: -18.555229677513733
- type: nauc_ndcg_at_20_diff1
value: 69.11315922808899
- type: nauc_ndcg_at_20_max
value: 40.3274581261148
- type: nauc_ndcg_at_20_std
value: -16.715264194903977
- type: nauc_ndcg_at_3_diff1
value: 70.10282801239877
- type: nauc_ndcg_at_3_max
value: 39.386237853059455
- type: nauc_ndcg_at_3_std
value: -17.43292542019076
- type: nauc_ndcg_at_5_diff1
value: 69.68625216343801
- type: nauc_ndcg_at_5_max
value: 39.56277357024661
- type: nauc_ndcg_at_5_std
value: -17.725437947649922
- type: nauc_precision_at_1000_diff1
value: 63.415227201991634
- type: nauc_precision_at_1000_max
value: 88.57862589480224
- type: nauc_precision_at_1000_std
value: 74.81423124805478
- type: nauc_precision_at_100_diff1
value: 60.96049530923491
- type: nauc_precision_at_100_max
value: 77.59214797029922
- type: nauc_precision_at_100_std
value: 30.640589569160998
- type: nauc_precision_at_10_diff1
value: 61.20085715681732
- type: nauc_precision_at_10_max
value: 49.84422733734606
- type: nauc_precision_at_10_std
value: -15.577033595242392
- type: nauc_precision_at_1_diff1
value: 73.69020304197174
- type: nauc_precision_at_1_max
value: 30.80837706084608
- type: nauc_precision_at_1_std
value: -18.555229677513733
- type: nauc_precision_at_20_diff1
value: 60.90742553930686
- type: nauc_precision_at_20_max
value: 60.63611383089591
- type: nauc_precision_at_20_std
value: -7.843087710432468
- type: nauc_precision_at_3_diff1
value: 67.35259682005486
- type: nauc_precision_at_3_max
value: 47.067986490892906
- type: nauc_precision_at_3_std
value: -15.937908115770094
- type: nauc_precision_at_5_diff1
value: 65.60152994415934
- type: nauc_precision_at_5_max
value: 49.101297109265154
- type: nauc_precision_at_5_std
value: -16.239935363441386
- type: nauc_recall_at_1000_diff1
value: 63.41522720199182
- type: nauc_recall_at_1000_max
value: 88.5786258948022
- type: nauc_recall_at_1000_std
value: 74.81423124805448
- type: nauc_recall_at_100_diff1
value: 60.960495309234815
- type: nauc_recall_at_100_max
value: 77.5921479702992
- type: nauc_recall_at_100_std
value: 30.640589569160916
- type: nauc_recall_at_10_diff1
value: 61.20085715681736
- type: nauc_recall_at_10_max
value: 49.844227337346084
- type: nauc_recall_at_10_std
value: -15.577033595242442
- type: nauc_recall_at_1_diff1
value: 73.69020304197174
- type: nauc_recall_at_1_max
value: 30.80837706084608
- type: nauc_recall_at_1_std
value: -18.555229677513733
- type: nauc_recall_at_20_diff1
value: 60.90742553930707
- type: nauc_recall_at_20_max
value: 60.636113830896065
- type: nauc_recall_at_20_std
value: -7.843087710432366
- type: nauc_recall_at_3_diff1
value: 67.35259682005484
- type: nauc_recall_at_3_max
value: 47.067986490892814
- type: nauc_recall_at_3_std
value: -15.937908115770183
- type: nauc_recall_at_5_diff1
value: 65.60152994415942
- type: nauc_recall_at_5_max
value: 49.10129710926523
- type: nauc_recall_at_5_std
value: -16.239935363441273
- type: ndcg_at_1
value: 49.5
- type: ndcg_at_10
value: 64.098
- type: ndcg_at_100
value: 67.086
- type: ndcg_at_1000
value: 67.552
- type: ndcg_at_20
value: 65.72200000000001
- type: ndcg_at_3
value: 59.863
- type: ndcg_at_5
value: 62.553000000000004
- type: precision_at_1
value: 49.5
- type: precision_at_10
value: 7.8100000000000005
- type: precision_at_100
value: 0.9159999999999999
- type: precision_at_1000
value: 0.095
- type: precision_at_20
value: 4.2250000000000005
- type: precision_at_3
value: 22.3
- type: precision_at_5
value: 14.680000000000001
- type: recall_at_1
value: 49.5
- type: recall_at_10
value: 78.10000000000001
- type: recall_at_100
value: 91.60000000000001
- type: recall_at_1000
value: 95.19999999999999
- type: recall_at_20
value: 84.5
- type: recall_at_3
value: 66.9
- type: recall_at_5
value: 73.4
task:
type: Retrieval
tags:
- mteb