QueryGym
QueryGym Leaderboard
Reproducible benchmarks for LLM query reformulation.
← Datasets

News

beir-v1.0.0-trec-news
All results produced by QueryGym · fully reproducible!

120 (method × LLM × retriever) configurations evaluated on this dataset.
Click any row or the + button to expand. The three steps (reformulate → retrieve → evaluate) for that run appear inline.

Retriever
Model
Method
120 / 120 runs
best in column
Method LLM Retriever nDCG@10 R@100
csqe Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.46260.4812
methodcsqe llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
csqe Qwen2.5-72B-Instruct BM25 0.38610.4892
methodcsqe llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
csqe Qwen2.5-72B-Instruct SPLADE++ 0.38710.4548
methodcsqe llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
csqe Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.43600.5126
methodcsqe llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
csqe Qwen2.5-7B-Instruct BM25 0.45040.5795
methodcsqe llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
csqe Qwen2.5-7B-Instruct SPLADE++ 0.40790.4866
methodcsqe llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
csqe gpt-4.1 BGE-base-en-v1.5 0.46310.5075
methodcsqe llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
csqe gpt-4.1 BM25 0.47900.5909
methodcsqe llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
csqe gpt-4.1 SPLADE++ 0.45020.5018
methodcsqe llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
csqe gpt-4.1-nano BGE-base-en-v1.5 0.43510.4753
methodcsqe llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
csqe gpt-4.1-nano BM25 0.42710.5221
methodcsqe llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
csqe gpt-4.1-nano SPLADE++ 0.41930.4601
methodcsqe llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
genqr Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.44090.5023
methodgenqr llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr Qwen2.5-72B-Instruct BM25 0.40030.5838
methodgenqr llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
genqr Qwen2.5-72B-Instruct SPLADE++ 0.38080.4754
methodgenqr llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
genqr Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.45260.4886
methodgenqr llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr Qwen2.5-7B-Instruct BM25 0.42950.5580
methodgenqr llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
genqr Qwen2.5-7B-Instruct SPLADE++ 0.39500.4527
methodgenqr llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
genqr gpt-4.1 BGE-base-en-v1.5 0.46410.5089
methodgenqr llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr gpt-4.1 BM25 0.46470.6096
methodgenqr llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
genqr gpt-4.1 SPLADE++ 0.42560.4877
methodgenqr llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
genqr gpt-4.1-nano BGE-base-en-v1.5 0.45480.5134
methodgenqr llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr gpt-4.1-nano BM25 0.42510.5834
methodgenqr llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
genqr gpt-4.1-nano SPLADE++ 0.40930.4933
methodgenqr llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.45150.5136
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct BM25 0.40800.5923
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
genqr_ensemble Qwen2.5-72B-Instruct SPLADE++ 0.39630.5087
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.45890.5172
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct BM25 0.43670.6031
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
genqr_ensemble Qwen2.5-7B-Instruct SPLADE++ 0.40490.4814
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
genqr_ensemble gpt-4.1 BGE-base-en-v1.5 0.47480.5249
methodgenqr_ensemble llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr_ensemble gpt-4.1 BM25 0.48600.6293
methodgenqr_ensemble llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
genqr_ensemble gpt-4.1 SPLADE++ 0.44380.5053
methodgenqr_ensemble llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
genqr_ensemble gpt-4.1-nano BGE-base-en-v1.5 0.47190.5175
methodgenqr_ensemble llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
genqr_ensemble gpt-4.1-nano BM25 0.43490.6199
methodgenqr_ensemble llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
genqr_ensemble gpt-4.1-nano SPLADE++ 0.41980.4906
methodgenqr_ensemble llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
lamer Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.45120.4936
methodlamer llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
lamer Qwen2.5-72B-Instruct BM25 0.46770.6105
methodlamer llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
lamer Qwen2.5-72B-Instruct SPLADE++ 0.41610.4850
methodlamer llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
lamer Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.45170.4753
methodlamer llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
lamer Qwen2.5-7B-Instruct BM25 0.44240.5960
methodlamer llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
lamer Qwen2.5-7B-Instruct SPLADE++ 0.39670.4728
methodlamer llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
lamer gpt-4.1 BGE-base-en-v1.5 0.43670.4591
methodlamer llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
lamer gpt-4.1 BM25 0.47990.5960
methodlamer llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
lamer gpt-4.1 SPLADE++ 0.45200.4770
methodlamer llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
lamer gpt-4.1-nano BGE-base-en-v1.5 0.40600.4264
methodlamer llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
lamer gpt-4.1-nano BM25 0.43280.5575
methodlamer llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
lamer gpt-4.1-nano SPLADE++ 0.40120.4661
methodlamer llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
mugi Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.47320.5298
methodmugi llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
mugi Qwen2.5-72B-Instruct BM25 0.50090.5921
methodmugi llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
mugi Qwen2.5-72B-Instruct SPLADE++ 0.43940.4972
methodmugi llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
mugi Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.46480.5142
methodmugi llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
mugi Qwen2.5-7B-Instruct BM25 0.44360.5767
methodmugi llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
mugi Qwen2.5-7B-Instruct SPLADE++ 0.40010.4725
methodmugi llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
mugi gpt-4.1 BGE-base-en-v1.5 0.48980.5212
methodmugi llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
mugi gpt-4.1 BM25 0.51560.6075
methodmugi llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
mugi gpt-4.1 SPLADE++ 0.44220.5002
methodmugi llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
mugi gpt-4.1-nano BGE-base-en-v1.5 0.46960.5081
methodmugi llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
mugi gpt-4.1-nano BM25 0.47070.5873
methodmugi llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
mugi gpt-4.1-nano SPLADE++ 0.40720.4770
methodmugi llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
qa_expand Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.48420.4983
methodqa_expand llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
qa_expand Qwen2.5-72B-Instruct BM25 0.44740.5517
methodqa_expand llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
qa_expand Qwen2.5-72B-Instruct SPLADE++ 0.41680.4803
methodqa_expand llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
qa_expand Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.44060.4862
methodqa_expand llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
qa_expand Qwen2.5-7B-Instruct BM25 0.43400.5419
methodqa_expand llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
qa_expand Qwen2.5-7B-Instruct SPLADE++ 0.39100.4548
methodqa_expand llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
qa_expand gpt-4.1 BGE-base-en-v1.5 0.46970.4852
methodqa_expand llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
qa_expand gpt-4.1 BM25 0.45020.5608
methodqa_expand llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
qa_expand gpt-4.1 SPLADE++ 0.42660.4566
methodqa_expand llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
qa_expand gpt-4.1-nano BGE-base-en-v1.5 0.42710.4749
methodqa_expand llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
qa_expand gpt-4.1-nano BM25 0.43260.5487
methodqa_expand llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
qa_expand gpt-4.1-nano SPLADE++ 0.42270.4696
methodqa_expand llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.48570.5135
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.46810.5148
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.40700.4508
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct BM25 0.46750.5557
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct BM25 0.41720.5578
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct BM25 0.48070.6048
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (FS) Qwen2.5-72B-Instruct SPLADE++ 0.42380.4846
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (ZS) Qwen2.5-72B-Instruct SPLADE++ 0.40680.4700
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (COT) Qwen2.5-72B-Instruct SPLADE++ 0.40540.4627
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.42950.4584
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.45370.5067
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.46270.5133
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct BM25 0.43490.5616
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct BM25 0.47780.5842
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct BM25 0.45070.5561
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
Q2D (COT) Qwen2.5-7B-Instruct SPLADE++ 0.38310.4524
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (ZS) Qwen2.5-7B-Instruct SPLADE++ 0.40270.4812
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (FS) Qwen2.5-7B-Instruct SPLADE++ 0.41460.4917
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
Q2D (ZS) gpt-4.1 BGE-base-en-v1.5 0.47610.5108
methodQ2D (ZS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (COT) gpt-4.1 BGE-base-en-v1.5 0.43310.4763
methodQ2D (COT) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (FS) gpt-4.1 BGE-base-en-v1.5 0.47150.5157
methodQ2D (FS) llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (ZS) gpt-4.1 BM25 0.49800.5858
methodQ2D (ZS) llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
Q2D (COT) gpt-4.1 BM25 0.46560.5829
methodQ2D (COT) llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
Q2D (FS) gpt-4.1 BM25 0.48010.5842
methodQ2D (FS) llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
Q2D (ZS) gpt-4.1 SPLADE++ 0.45170.4786
methodQ2D (ZS) llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
Q2D (COT) gpt-4.1 SPLADE++ 0.41600.4741
methodQ2D (COT) llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
Q2D (FS) gpt-4.1 SPLADE++ 0.43020.5009
methodQ2D (FS) llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
Q2D (COT) gpt-4.1-nano BGE-base-en-v1.5 0.43120.4754
methodQ2D (COT) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (FS) gpt-4.1-nano BGE-base-en-v1.5 0.45390.4763
methodQ2D (FS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (ZS) gpt-4.1-nano BGE-base-en-v1.5 0.44670.4931
methodQ2D (ZS) llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
Q2D (COT) gpt-4.1-nano BM25 0.46010.5728
methodQ2D (COT) llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
Q2D (FS) gpt-4.1-nano BM25 0.44420.5398
methodQ2D (FS) llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
Q2D (ZS) gpt-4.1-nano BM25 0.46850.5564
methodQ2D (ZS) llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
Q2D (COT) gpt-4.1-nano SPLADE++ 0.40530.4554
methodQ2D (COT) llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
Q2D (FS) gpt-4.1-nano SPLADE++ 0.42920.4573
methodQ2D (FS) llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
Q2D (ZS) gpt-4.1-nano SPLADE++ 0.40550.4651
methodQ2D (ZS) llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.
query2e Qwen2.5-72B-Instruct BGE-base-en-v1.5 0.45090.5067
methodquery2e llmQwen2.5-72B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
query2e Qwen2.5-72B-Instruct BM25 0.44840.5647
methodquery2e llmQwen2.5-72B-Instruct retrieverBM25 datasetNews
No run config available.
query2e Qwen2.5-72B-Instruct SPLADE++ 0.40760.4799
methodquery2e llmQwen2.5-72B-Instruct retrieverSPLADE++ datasetNews
No run config available.
query2e Qwen2.5-7B-Instruct BGE-base-en-v1.5 0.44540.4967
methodquery2e llmQwen2.5-7B-Instruct retrieverBGE-base-en-v1.5 datasetNews
No run config available.
query2e Qwen2.5-7B-Instruct BM25 0.45030.5824
methodquery2e llmQwen2.5-7B-Instruct retrieverBM25 datasetNews
No run config available.
query2e Qwen2.5-7B-Instruct SPLADE++ 0.40730.4888
methodquery2e llmQwen2.5-7B-Instruct retrieverSPLADE++ datasetNews
No run config available.
query2e gpt-4.1 BGE-base-en-v1.5 0.44480.4848
methodquery2e llmgpt-4.1 retrieverBGE-base-en-v1.5 datasetNews
No run config available.
query2e gpt-4.1 BM25 0.46330.5807
methodquery2e llmgpt-4.1 retrieverBM25 datasetNews
No run config available.
query2e gpt-4.1 SPLADE++ 0.42060.4992
methodquery2e llmgpt-4.1 retrieverSPLADE++ datasetNews
No run config available.
query2e gpt-4.1-nano BGE-base-en-v1.5 0.45040.5018
methodquery2e llmgpt-4.1-nano retrieverBGE-base-en-v1.5 datasetNews
No run config available.
query2e gpt-4.1-nano BM25 0.45570.5827
methodquery2e llmgpt-4.1-nano retrieverBM25 datasetNews
No run config available.
query2e gpt-4.1-nano SPLADE++ 0.40860.4906
methodquery2e llmgpt-4.1-nano retrieverSPLADE++ datasetNews
No run config available.