QueryGym
QueryGym Leaderboard
Reproducible benchmarks for LLM query reformulation.
← Datasets

SciFact

beir-v1.0.0-scifact
All results produced by QueryGym · fully reproducible!

120 (method × LLM × retriever) configurations evaluated on this dataset.
Click any row or the + button to expand. The three steps (reformulate → retrieve → evaluate) for that run appear inline.

Retriever
Model
Method
120 / 120 runs
best in column
Method LLM Retriever nDCG@10 R@100
csqe Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.74840.9667
methodcsqe llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
csqe Qwen2.5-72B-Instruct BM25 0.7141—
methodcsqe llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
csqe Qwen2.5-72B-Instruct SPLADE++ 0.69660.9433
methodcsqe llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
csqe Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.74150.9727
methodcsqe llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
csqe Qwen2.5-7B-Instruct BM25 0.71830.9543
methodcsqe llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
csqe Qwen2.5-7B-Instruct SPLADE++ 0.67650.9527
methodcsqe llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
csqe gpt-4.1 BGE-base-en-v1.5 0.75530.9633
methodcsqe llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
csqe gpt-4.1 BM25 0.72060.9487
methodcsqe llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
csqe gpt-4.1 SPLADE++ 0.70650.9593
methodcsqe llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
csqe gpt-4.1-nano BGE-base-en-v1.5 0.75830.9600
methodcsqe llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
csqe gpt-4.1-nano BM25 0.70990.9587
methodcsqe llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
csqe gpt-4.1-nano SPLADE++ 0.70550.9533
methodcsqe llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
genqr Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.73390.9650
methodgenqr llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr Qwen2.5-72B-Instruct BM25 0.6976—
methodgenqr llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
genqr Qwen2.5-72B-Instruct SPLADE++ 0.74680.9413
methodgenqr llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
genqr Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.72540.9600
methodgenqr llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr Qwen2.5-7B-Instruct BM25 0.69190.9413
methodgenqr llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
genqr Qwen2.5-7B-Instruct SPLADE++ 0.69420.9297
methodgenqr llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
genqr gpt-4.1 BGE-base-en-v1.5 0.74800.9700
methodgenqr llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr gpt-4.1 BM25 0.72620.9632
methodgenqr llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
genqr gpt-4.1 SPLADE++ 0.72770.9500
methodgenqr llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
genqr gpt-4.1-nano BGE-base-en-v1.5 0.75530.9633
methodgenqr llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr gpt-4.1-nano BM25 0.70110.9566
methodgenqr llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
genqr gpt-4.1-nano SPLADE++ 0.71840.9633
methodgenqr llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.74960.9700
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BM25 0.7089—
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct SPLADE++ 0.71350.9433
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.73750.9667
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BM25 0.70350.9476
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct SPLADE++ 0.69640.9460
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
genqr_ensemble gpt-4.1 BGE-base-en-v1.5 0.75890.9700
methodgenqr_ensemble llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr_ensemble gpt-4.1 BM25 0.72510.9666
methodgenqr_ensemble llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
genqr_ensemble gpt-4.1 SPLADE++ 0.71750.9433
methodgenqr_ensemble llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
genqr_ensemble gpt-4.1-nano BGE-base-en-v1.5 0.74690.9633
methodgenqr_ensemble llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
genqr_ensemble gpt-4.1-nano BM25 0.70340.9626
methodgenqr_ensemble llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
genqr_ensemble gpt-4.1-nano SPLADE++ 0.71580.9560
methodgenqr_ensemble llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
lamer Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.75240.9800
methodlamer llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
lamer Qwen2.5-72B-Instruct BM25 0.7251—
methodlamer llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
lamer Qwen2.5-72B-Instruct SPLADE++ 0.70460.9600
methodlamer llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
lamer Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.74660.9733
methodlamer llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
lamer Qwen2.5-7B-Instruct BM25 0.71400.9593
methodlamer llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
lamer Qwen2.5-7B-Instruct SPLADE++ 0.66510.9560
methodlamer llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
lamer gpt-4.1 BGE-base-en-v1.5 0.75720.9733
methodlamer llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
lamer gpt-4.1 BM25 0.72530.9487
methodlamer llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
lamer gpt-4.1 SPLADE++ 0.71820.9577
methodlamer llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
lamer gpt-4.1-nano BGE-base-en-v1.5 0.75070.9593
methodlamer llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
lamer gpt-4.1-nano BM25 0.72200.9393
methodlamer llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
lamer gpt-4.1-nano SPLADE++ 0.72070.9443
methodlamer llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
mugi Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.74530.9700
methodmugi llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
mugi Qwen2.5-72B-Instruct BM25 0.7203—
methodmugi llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
mugi Qwen2.5-72B-Instruct SPLADE++ 0.69510.9493
methodmugi llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
mugi Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.74490.9767
methodmugi llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
mugi Qwen2.5-7B-Instruct BM25 0.70630.9627
methodmugi llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
mugi Qwen2.5-7B-Instruct SPLADE++ 0.66650.9593
methodmugi llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
mugi gpt-4.1 BGE-base-en-v1.5 0.75690.9767
methodmugi llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
mugi gpt-4.1 BM25 0.73450.9660
methodmugi llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
mugi gpt-4.1 SPLADE++ 0.70590.9600
methodmugi llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
mugi gpt-4.1-nano BGE-base-en-v1.5 0.74570.9800
methodmugi llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
mugi gpt-4.1-nano BM25 0.73180.9627
methodmugi llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
mugi gpt-4.1-nano SPLADE++ 0.69000.9527
methodmugi llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
qa_expand Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.74310.9667
methodqa_expand llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
qa_expand Qwen2.5-72B-Instruct BM25 0.7015—
methodqa_expand llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
qa_expand Qwen2.5-72B-Instruct SPLADE++ 0.67960.9393
methodqa_expand llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
qa_expand Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.74340.9583
methodqa_expand llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
qa_expand Qwen2.5-7B-Instruct BM25 0.68570.9347
methodqa_expand llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
qa_expand Qwen2.5-7B-Instruct SPLADE++ 0.66160.9547
methodqa_expand llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
qa_expand gpt-4.1 BGE-base-en-v1.5 0.73670.9600
methodqa_expand llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
qa_expand gpt-4.1 BM25 0.70630.9403
methodqa_expand llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
qa_expand gpt-4.1 SPLADE++ 0.69640.9493
methodqa_expand llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
qa_expand gpt-4.1-nano BGE-base-en-v1.5 0.74860.9593
methodqa_expand llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
qa_expand gpt-4.1-nano BM25 0.70590.9430
methodqa_expand llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
qa_expand gpt-4.1-nano SPLADE++ 0.69390.9420
methodqa_expand llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.75400.9633
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.74940.9667
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.73870.9600
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BM25 0.7172—
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BM25 0.7077—
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BM25 0.7163—
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct SPLADE++ 0.70350.9567
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct SPLADE++ 0.69650.9560
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct SPLADE++ 0.68340.9533
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.73360.9667
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.75200.9633
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.74540.9567
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BM25 0.70960.9427
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BM25 0.71490.9443
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BM25 0.70420.9443
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct SPLADE++ 0.68250.9467
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct SPLADE++ 0.68030.9567
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct SPLADE++ 0.71200.9500
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1 BGE-base-en-v1.5 0.76090.9633
methodQ2D (ZS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (COT) gpt-4.1 BGE-base-en-v1.5 0.75800.9633
methodQ2D (COT) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (FS) gpt-4.1 BGE-base-en-v1.5 0.75190.9667
methodQ2D (FS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1 BM25 0.72030.9477
methodQ2D (ZS) llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
Q2D (COT) gpt-4.1 BM25 0.71350.9510
methodQ2D (COT) llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
Q2D (FS) gpt-4.1 BM25 0.71230.9493
methodQ2D (FS) llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1 SPLADE++ 0.70350.9553
methodQ2D (ZS) llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (COT) gpt-4.1 SPLADE++ 0.71200.9460
methodQ2D (COT) llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (FS) gpt-4.1 SPLADE++ 0.70930.9567
methodQ2D (FS) llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (COT) gpt-4.1-nano BGE-base-en-v1.5 0.74990.9633
methodQ2D (COT) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (FS) gpt-4.1-nano BGE-base-en-v1.5 0.74170.9567
methodQ2D (FS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1-nano BGE-base-en-v1.5 0.75410.9633
methodQ2D (ZS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
Q2D (COT) gpt-4.1-nano BM25 0.72730.9560
methodQ2D (COT) llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
Q2D (FS) gpt-4.1-nano BM25 0.70530.9410
methodQ2D (FS) llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1-nano BM25 0.71700.9403
methodQ2D (ZS) llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
Q2D (COT) gpt-4.1-nano SPLADE++ 0.70650.9433
methodQ2D (COT) llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (FS) gpt-4.1-nano SPLADE++ 0.71210.9400
methodQ2D (FS) llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
Q2D (ZS) gpt-4.1-nano SPLADE++ 23.00000.9493
methodQ2D (ZS) llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.
query2e Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.73820.9567
methodquery2e llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
query2e Qwen2.5-72B-Instruct BM25 0.6969—
methodquery2e llmQwen2.5-72B-Instruct retrieverBM25 datasetSciFact
No run config available.
query2e Qwen2.5-72B-Instruct SPLADE++ 0.70490.9427
methodquery2e llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
query2e Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.73780.9633
methodquery2e llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
query2e Qwen2.5-7B-Instruct BM25 0.69670.9520
methodquery2e llmQwen2.5-7B-Instruct retrieverBM25 datasetSciFact
No run config available.
query2e Qwen2.5-7B-Instruct SPLADE++ 0.71150.9493
methodquery2e llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetSciFact
No run config available.
query2e gpt-4.1 BGE-base-en-v1.5 0.74170.9633
methodquery2e llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
query2e gpt-4.1 BM25 0.70890.9403
methodquery2e llmgpt-4.1 retrieverBM25 datasetSciFact
No run config available.
query2e gpt-4.1 SPLADE++ 0.71870.9393
methodquery2e llmgpt-4.1 retrieverSPLADE++ datasetSciFact
No run config available.
query2e gpt-4.1-nano BGE-base-en-v1.5 0.74770.9633
methodquery2e llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetSciFact
No run config available.
query2e gpt-4.1-nano BM25 0.70160.9480
methodquery2e llmgpt-4.1-nano retrieverBM25 datasetSciFact
No run config available.
query2e gpt-4.1-nano SPLADE++ 0.72060.9387
methodquery2e llmgpt-4.1-nano retrieverSPLADE++ datasetSciFact
No run config available.