QueryGym
QueryGym Leaderboard
Reproducible benchmarks for LLM query reformulation.
← Datasets

COVID

beir-v1.0.0-trec-covid
All results produced by QueryGym · fully reproducible!

120 (method × LLM × retriever) configurations evaluated on this dataset.
Click any row or the + button to expand. The three steps (reformulate → retrieve → evaluate) for that run appear inline.

Retriever
Model
Method
120 / 120 runs
best in column
Method LLM Retriever nDCG@10 R@100
csqe Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.77930.1410
methodcsqe llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
csqe Qwen2.5-72B-Instruct BM25 0.67160.1491
methodcsqe llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
csqe Qwen2.5-72B-Instruct SPLADE++ 0.61180.1082
methodcsqe llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
csqe Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.78620.1449
methodcsqe llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
csqe Qwen2.5-7B-Instruct BM25 0.67570.1600
methodcsqe llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
csqe Qwen2.5-7B-Instruct SPLADE++ 0.60960.1024
methodcsqe llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
csqe gpt-4.1 BGE-base-en-v1.5 0.78790.1431
methodcsqe llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
csqe gpt-4.1 BM25 0.69940.1638
methodcsqe llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
csqe gpt-4.1 SPLADE++ 0.68110.1116
methodcsqe llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
csqe gpt-4.1-nano BGE-base-en-v1.5 0.81740.1442
methodcsqe llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
csqe gpt-4.1-nano BM25 0.61710.1543
methodcsqe llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
csqe gpt-4.1-nano SPLADE++ 0.63130.1132
methodcsqe llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
genqr Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.78690.1416
methodgenqr llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr Qwen2.5-72B-Instruct BM25 0.61290.1349
methodgenqr llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
genqr Qwen2.5-72B-Instruct SPLADE++ 0.62920.1055
methodgenqr llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
genqr Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.76080.1382
methodgenqr llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr Qwen2.5-7B-Instruct BM25 0.65230.1522
methodgenqr llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
genqr Qwen2.5-7B-Instruct SPLADE++ 0.70600.1263
methodgenqr llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
genqr gpt-4.1 BGE-base-en-v1.5 0.77840.1475
methodgenqr llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr gpt-4.1 BM25 0.68690.1627
methodgenqr llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
genqr gpt-4.1 SPLADE++ 0.68200.1193
methodgenqr llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
genqr gpt-4.1-nano BGE-base-en-v1.5 0.79870.1440
methodgenqr llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr gpt-4.1-nano BM25 0.66620.1561
methodgenqr llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
genqr gpt-4.1-nano SPLADE++ 0.65940.1163
methodgenqr llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.79150.1407
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BM25 0.64370.1451
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct SPLADE++ 0.61620.1099
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.77540.1379
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BM25 0.67800.1745
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct SPLADE++ 0.64200.1117
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
genqr_ensemble gpt-4.1 BGE-base-en-v1.5 0.79990.1443
methodgenqr_ensemble llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr_ensemble gpt-4.1 BM25 0.75280.1839
methodgenqr_ensemble llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
genqr_ensemble gpt-4.1 SPLADE++ 0.67310.1198
methodgenqr_ensemble llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
genqr_ensemble gpt-4.1-nano BGE-base-en-v1.5 0.79760.1425
methodgenqr_ensemble llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
genqr_ensemble gpt-4.1-nano BM25 0.68840.1690
methodgenqr_ensemble llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
genqr_ensemble gpt-4.1-nano SPLADE++ 0.65140.1166
methodgenqr_ensemble llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
lamer Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.79410.1401
methodlamer llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
lamer Qwen2.5-72B-Instruct BM25 0.72400.1667
methodlamer llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
lamer Qwen2.5-72B-Instruct SPLADE++ 0.65430.1057
methodlamer llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
lamer Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.78430.1360
methodlamer llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
lamer Qwen2.5-7B-Instruct BM25 0.69550.1704
methodlamer llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
lamer Qwen2.5-7B-Instruct SPLADE++ 0.63390.1002
methodlamer llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
lamer gpt-4.1 BGE-base-en-v1.5 0.77960.1373
methodlamer llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
lamer gpt-4.1 BM25 0.70200.1661
methodlamer llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
lamer gpt-4.1 SPLADE++ 0.63120.1081
methodlamer llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
lamer gpt-4.1-nano BGE-base-en-v1.5 0.80070.1340
methodlamer llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
lamer gpt-4.1-nano BM25 0.67210.1748
methodlamer llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
lamer gpt-4.1-nano SPLADE++ 0.62850.1143
methodlamer llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
mugi Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.79720.1425
methodmugi llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
mugi Qwen2.5-72B-Instruct BM25 0.69270.1694
methodmugi llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
mugi Qwen2.5-72B-Instruct SPLADE++ 0.66390.1105
methodmugi llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
mugi Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.80710.1406
methodmugi llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
mugi Qwen2.5-7B-Instruct BM25 0.67710.1628
methodmugi llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
mugi Qwen2.5-7B-Instruct SPLADE++ 0.65470.1045
methodmugi llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
mugi gpt-4.1 BGE-base-en-v1.5 0.80240.1427
methodmugi llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
mugi gpt-4.1 BM25 0.71370.1739
methodmugi llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
mugi gpt-4.1 SPLADE++ 0.64580.1118
methodmugi llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
mugi gpt-4.1-nano BGE-base-en-v1.5 0.79800.1425
methodmugi llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
mugi gpt-4.1-nano BM25 0.70620.1713
methodmugi llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
mugi gpt-4.1-nano SPLADE++ 0.63170.1144
methodmugi llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
qa_expand Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.77750.1370
methodqa_expand llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
qa_expand Qwen2.5-72B-Instruct BM25 0.68090.1600
methodqa_expand llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
qa_expand Qwen2.5-72B-Instruct SPLADE++ 0.63240.1079
methodqa_expand llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
qa_expand Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.76680.1378
methodqa_expand llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
qa_expand Qwen2.5-7B-Instruct BM25 0.67290.1569
methodqa_expand llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
qa_expand Qwen2.5-7B-Instruct SPLADE++ 0.64310.1103
methodqa_expand llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
qa_expand gpt-4.1 BGE-base-en-v1.5 0.79540.1419
methodqa_expand llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
qa_expand gpt-4.1 BM25 0.70650.1620
methodqa_expand llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
qa_expand gpt-4.1 SPLADE++ 0.69410.1152
methodqa_expand llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
qa_expand gpt-4.1-nano BGE-base-en-v1.5 0.74890.1355
methodqa_expand llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
qa_expand gpt-4.1-nano BM25 0.68850.1583
methodqa_expand llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
qa_expand gpt-4.1-nano SPLADE++ 0.70790.1215
methodqa_expand llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.78910.1401
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.77120.1382
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.77100.1367
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BM25 0.69730.1672
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BM25 0.67850.1590
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BM25 0.70780.1673
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct SPLADE++ 0.66890.1142
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct SPLADE++ 0.62720.1095
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct SPLADE++ 0.64250.1159
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.77690.1386
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.82200.1440
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.79220.1388
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BM25 0.69970.1620
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BM25 0.74230.1668
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BM25 0.70710.1628
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct SPLADE++ 0.65670.1147
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct SPLADE++ 0.66730.1124
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct SPLADE++ 0.67930.1095
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1 BGE-base-en-v1.5 0.80610.1454
methodQ2D (ZS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (COT) gpt-4.1 BGE-base-en-v1.5 0.79840.1380
methodQ2D (COT) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (FS) gpt-4.1 BGE-base-en-v1.5 0.80390.1411
methodQ2D (FS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1 BM25 0.74300.1704
methodQ2D (ZS) llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
Q2D (COT) gpt-4.1 BM25 0.72770.1696
methodQ2D (COT) llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
Q2D (FS) gpt-4.1 BM25 0.70810.1639
methodQ2D (FS) llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1 SPLADE++ 0.63400.1089
methodQ2D (ZS) llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (COT) gpt-4.1 SPLADE++ 0.68580.1056
methodQ2D (COT) llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (FS) gpt-4.1 SPLADE++ 0.65910.1099
methodQ2D (FS) llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (COT) gpt-4.1-nano BGE-base-en-v1.5 0.79950.1420
methodQ2D (COT) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (FS) gpt-4.1-nano BGE-base-en-v1.5 0.77930.1402
methodQ2D (FS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1-nano BGE-base-en-v1.5 0.80190.1417
methodQ2D (ZS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
Q2D (COT) gpt-4.1-nano BM25 0.75030.1744
methodQ2D (COT) llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
Q2D (FS) gpt-4.1-nano BM25 0.68270.1634
methodQ2D (FS) llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1-nano BM25 0.69670.1656
methodQ2D (ZS) llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
Q2D (COT) gpt-4.1-nano SPLADE++ 0.68090.1163
methodQ2D (COT) llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (FS) gpt-4.1-nano SPLADE++ 0.67150.1182
methodQ2D (FS) llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
Q2D (ZS) gpt-4.1-nano SPLADE++ 0.66450.1146
methodQ2D (ZS) llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.
query2e Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.78570.1412
methodquery2e llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
query2e Qwen2.5-72B-Instruct BM25 0.69420.1611
methodquery2e llmQwen2.5-72B-Instruct retrieverBM25 datasetCOVID
No run config available.
query2e Qwen2.5-72B-Instruct SPLADE++ 0.61960.1201
methodquery2e llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
query2e Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.76180.1379
methodquery2e llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
query2e Qwen2.5-7B-Instruct BM25 0.69450.1653
methodquery2e llmQwen2.5-7B-Instruct retrieverBM25 datasetCOVID
No run config available.
query2e Qwen2.5-7B-Instruct SPLADE++ 0.60800.1093
methodquery2e llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetCOVID
No run config available.
query2e gpt-4.1 BGE-base-en-v1.5 0.77410.1404
methodquery2e llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
query2e gpt-4.1 BM25 0.71500.1772
methodquery2e llmgpt-4.1 retrieverBM25 datasetCOVID
No run config available.
query2e gpt-4.1 SPLADE++ 0.68690.1222
methodquery2e llmgpt-4.1 retrieverSPLADE++ datasetCOVID
No run config available.
query2e gpt-4.1-nano BGE-base-en-v1.5 0.78030.1407
methodquery2e llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetCOVID
No run config available.
query2e gpt-4.1-nano BM25 0.73730.1765
methodquery2e llmgpt-4.1-nano retrieverBM25 datasetCOVID
No run config available.
query2e gpt-4.1-nano SPLADE++ 0.67470.1214
methodquery2e llmgpt-4.1-nano retrieverSPLADE++ datasetCOVID
No run config available.