SPLADE++

learned_sparse

All results produced by QueryGym · fully reproducible!

40 method × LLM configurations using this retriever across BEIR, MS MARCO DL, and DL-HARD.
Click any row or the + button to expand. Tabs switch dataset context. The three steps (reformulate → retrieve → evaluate) update accordingly.

	Method	LLM	ArguAna		DBPedia		FiQA		SciFact		COVID		News		BRIGHT — AOPS		BRIGHT — Biology		BRIGHT — Earth Science		BRIGHT — Economics		BRIGHT — LeetCode		BRIGHT — Pony		BRIGHT — Psychology		BRIGHT — Robotics		BRIGHT — Stack Overflow		BRIGHT — Sustainable Living		BRIGHT — TheoremQA Questions		BRIGHT — TheoremQA Theorems		DL-HARD		DL 2019		DL 2020
	Method	LLM	nDCG@10	R@100	nDCG@10	R@100	nDCG@10	R@100	nDCG@10	R@100	nDCG@10	R@100	nDCG@10	R@100																									nDCG@10	R@1k	nDCG@10	R@1k	nDCG@10	R@1k
	csqe	Qwen2.5-72B-Instruct	0.5118	0.9787	0.3686	0.5021	0.3075	0.6521	0.6966	0.9433	0.6118	0.1082	0.3871	0.4548	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2857	0.8246	0.6189	0.9070	0.5736	0.9052
methodcsqe llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 43c9e25451d58e11 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 49c7af4b43926798 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 868b26500a878476 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id dfae7d6c06e355d5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 92ba76bc29a9d68f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 979e9554baa56930 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id c106bbf40f6cf8e6 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 8057160502ca9c91 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method csqe \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id f616925e2ae1ece5 · open full run detail →
	csqe	Qwen2.5-7B-Instruct	0.5100	0.9801	0.3661	0.4830	0.3035	0.6521	0.6765	0.9527	0.6096	0.1024	0.4079	0.4866	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3025	0.8057	0.6523	0.9089	0.6164	0.9039
methodcsqe llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id e30beb7d6120032b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id c645b63d148df699 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 7c0e15a8a3ed05d7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id d991a771a4ec16c3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 6a5b94d82f568fbb · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id bfcb1e13cc699a61 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 262563ce22f62374 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 38aa528bf5037635 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method csqe \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 0947529629fd12ec · open full run detail →
	csqe	gpt-4.1	0.3801	0.9829	0.3962	0.5232	0.3294	0.6748	0.7065	0.9593	0.6811	0.1116	0.4502	0.5018	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3690	0.8341	0.6936	0.9193	0.6796	0.9397
methodcsqe llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 3e383ae339c3a62e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id b3f2134d9575311c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id f5ef9755a807ae98 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id b05b8be9006ec3a1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 8be5974aec2d4609 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 7017e4a12692cfa0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 322e4bb8fbbdbe9b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 6148ce92e9e11b2d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method csqe \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 435cc0da60103e45 · open full run detail →
	csqe	gpt-4.1-nano	0.3792	0.9801	0.3805	0.5235	0.3256	0.6702	0.7055	0.9533	0.6313	0.1132	0.4193	0.4601	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2789	0.7872	0.6134	0.8900	0.5883	0.9119
methodcsqe llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id e0f4947fbccc6397 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 92ded4a648735d6b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 481dd45b5c62d23c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 49cb755c774c5dbb · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id f980832298d33f40 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id b82e8af82f8b212c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id cc25c04a032109ab · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 7a7bacfd287897cd · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method csqe \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id e646c2b1b81107fa · open full run detail →
	genqr	Qwen2.5-72B-Instruct	0.5201	0.9815	0.3579	0.5275	0.2868	0.6217	0.7468	0.9413	0.6292	0.1055	0.3808	0.4754	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2916	0.7861	0.6154	0.9030	0.5751	0.8971
methodgenqr llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 148aa91670cef258 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 5fff72e0044be6ce · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id a206750947de6c91 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id b884275c4c2c7fd3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 9f27e2872fe1496f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 95e306b89d6cccd8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id d52ede3b6a3b2d5e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 6c59db025fdb4940 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 1f8ed462c5bf6734 · open full run detail →
	genqr	Qwen2.5-7B-Instruct	0.5211	0.9851	0.3703	0.5386	0.3057	0.6309	0.6942	0.9297	0.7060	0.1263	0.3950	0.4527	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3386	0.8000	0.6449	0.8870	0.6115	0.8989
methodgenqr llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 5490e12acba323e4 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 9f6894dc79d6fc31 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 284c21e5d24c5131 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 0964b93d4ba47353 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 6b168618c3402fbc · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id e24196fcf6955d28 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 91bac4d1f65b2f8b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id c56507d9c77300dd · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 303a213cc7bd8d75 · open full run detail →
	genqr	gpt-4.1	0.3755	0.9836	0.3827	0.5414	0.3243	0.6774	0.7277	0.9500	0.6820	0.1193	0.4256	0.4877	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3800	0.8488	0.7065	0.9333	0.6260	0.9143
methodgenqr llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 4302aab08a4f7188 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 4fcf6543e54d9299 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 2411a8a37f38939d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 49f8ea07cc69238a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id ffa0a9919989ff90 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 3c8379a7cbd71498 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 7f9873b6ad2a9120 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 63ce7820d8cbb9e9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id ab6369ce9b83adec · open full run detail →
	genqr	gpt-4.1-nano	0.3773	0.9829	0.3592	0.5267	0.3025	0.6466	0.7184	0.9633	0.6594	0.1163	0.4093	0.4933	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3043	0.8408	0.6351	0.9162	0.6011	0.9074
methodgenqr llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id abd5781879e192f3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id a43a25b7bc2e4d33 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id bded8e04d25e1a18 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 2cd203ab13b88284 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id da510807270c6d4e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 4875e088cdf4e87d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id d63a50faa6ada82d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 7bd26a175c531f17 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id eb4797ccddea3ab9 · open full run detail →
	genqr_ensemble	Qwen2.5-72B-Instruct	0.5193	0.9822	0.4271	0.5565	0.3062	0.6136	0.7135	0.9433	0.6162	0.1099	0.3963	0.5087	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2849	0.7823	0.5979	0.9053	0.5447	0.8886
methodgenqr_ensemble llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id b5d7e1f37ccc765c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id b0770923fff53c8c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 1e905171b4723501 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 5e2a2a7f7ee8b12a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id e961b4265e038078 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 805d3ad7780ed870 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id df7dd68f00db9700 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 82bd5e7500a6859b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr_ensemble \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 8c88fc4b9ce35ca0 · open full run detail →
	genqr_ensemble	Qwen2.5-7B-Instruct	0.5180	0.9815	0.3589	0.5194	0.2882	0.6249	0.6964	0.9460	0.6420	0.1117	0.4049	0.4814	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3292	0.8005	0.5948	0.8824	0.6307	0.9020
methodgenqr_ensemble llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id aa0a09c2d60fb379 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 0952aa4806a53daa · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 0f7fecda7817ce82 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id ec9647c785553bac · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 23d94602e1b554fc · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id f03a3866e19eaf1b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id e525464e0a9d8b0c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 720cb5b5982a9dc3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr_ensemble \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id d3c9513f7c3251ba · open full run detail →
	genqr_ensemble	gpt-4.1	0.3806	0.9808	0.3643	0.5365	0.3014	0.6536	0.7175	0.9433	0.6731	0.1198	0.4438	0.5053	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3047	0.8207	0.6859	0.9020	0.5857	0.9141
methodgenqr_ensemble llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 7a007f087dc3fe3a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id f87d64a2a96f02ce · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id d3000ae587a47dae · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id c5b0b0b9101d0743 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 3731009953d5205d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 93ca403b7b7e40d1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 0aa4af8b129e8cac · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 804db777441d096e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr_ensemble \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 0c8064c7d5973165 · open full run detail →
	genqr_ensemble	gpt-4.1-nano	0.3818	0.9808	0.3611	0.5276	0.2891	0.6311	0.7158	0.9560	0.6514	0.1166	0.4198	0.4906	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3233	0.8400	0.6617	0.9104	0.6044	0.9194
methodgenqr_ensemble llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id dc6d84f6cbc27959 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 598f90bace200e3f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 0efa2c908a346cfc · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 1ffdc6dd8afdbde9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 15f22a7ac3946768 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 82c8eaa7333bea7e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id fd936b5ba2d8436c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 15d4d1caa3705208 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method genqr_ensemble \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 05024db94c86eaf9 · open full run detail →
	lamer	Qwen2.5-72B-Instruct	0.5161	0.9815	0.3697	0.4883	0.3041	0.6516	0.7046	0.9600	0.6543	0.1057	0.4161	0.4850	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3648	0.8156	0.6651	0.8956	0.6483	0.9195
methodlamer llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 579cc5a3039bc56f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 57940bb11cb19219 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 6c3920399824ba8b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id d4f19d91c62f2f77 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id f72e663204744842 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id ee987027e7b6ca46 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id d207cfd370d5d82c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 41df03a4d84da054 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method lamer \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id c45cd567ed787d00 · open full run detail →
	lamer	Qwen2.5-7B-Instruct	0.5148	0.9794	0.3499	0.4799	0.2944	0.6487	0.6651	0.9560	0.6339	0.1002	0.3967	0.4728	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3280	0.7917	0.6465	0.8654	0.6076	0.9213
methodlamer llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 0616194d13348b61 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 327fda84b730628b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 6e3f251e13c96034 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 13886968f236369d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id e760de4c23be40a5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 0fd33ea475108659 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id dbee4e6ccdf98310 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 66711eb3e485ecb1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method lamer \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id ebdfca59bc737639 · open full run detail →
	lamer	gpt-4.1	0.3836	0.9829	0.3559	0.4904	0.3292	0.6724	0.7182	0.9577	0.6312	0.1081	0.4520	0.4770	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3673	0.8246	0.6836	0.9065	0.6390	0.9378
methodlamer llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 8a03d57b02937b1f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 705d1c4e8e7c517d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id a1dea4fa8242d0c9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 62108cd5b314e124 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 031a4a3df33cc5ba · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id d0ecf5c9f11b78a0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 7b90f42881e20399 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id b70672ae7d83c378 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method lamer \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 44018304f3ecf315 · open full run detail →
	lamer	gpt-4.1-nano	0.3800	0.9780	0.3316	0.4680	0.3014	0.6543	0.7207	0.9443	0.6285	0.1143	0.4012	0.4661	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3459	0.7969	0.6916	0.8975	0.6254	0.9244
methodlamer llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 5f42ef0d9c5eedd2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 03cecba09b8621cc · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 217166f1520b6b38 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id c213988750249688 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id a7f15b7e6404eda3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id a01865b479e96df5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id bf4759de8f4f3141 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id a4920606bfbad73e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method lamer \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id ece61fc263f8c089 · open full run detail →
	mugi	Qwen2.5-72B-Instruct	0.5031	0.9787	0.3735	0.5044	0.3023	0.6787	0.6951	0.9493	0.6639	0.1105	0.4394	0.4972	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3260	0.8098	0.6746	0.9275	0.6419	0.9165
methodmugi llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id c5cd4658373a61c8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id bbed6bd9f822ca4b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 5328e402f08642b8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 9f31850bdb47fef0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id d5d4c66acd86484c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 6afab8749e67e7e2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 00665889c6151a55 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id d5b50b564ccc29e8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method mugi \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 2b1068b944df9dec · open full run detail →
	mugi	Qwen2.5-7B-Instruct	0.5101	0.9787	0.3600	0.4989	0.2953	0.6597	0.6665	0.9593	0.6547	0.1045	0.4001	0.4725	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2642	0.8028	0.5773	0.8929	0.5527	0.9104
methodmugi llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 1543aaf2b16b0616 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 35d99cd527f8f873 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 9a3a2408466e6ceb · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 1a50c50e259858ab · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 266e1d1425d0563f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 8d15e8f1910cdbc3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id a3c8453a6b8584b7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id e0b6047be3d4daf2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method mugi \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 57db88c60c930ee5 · open full run detail →
	mugi	gpt-4.1	0.3703	0.9780	0.3843	0.5137	0.3352	0.6799	0.7059	0.9600	0.6458	0.1118	0.4422	0.5002	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3625	0.8111	0.6859	0.9088	0.6508	0.9199
methodmugi llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id e3b7f0dd5a400a85 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id a263942c47cbce72 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 2a3dc4da10dec081 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 32603c5fb254483d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 35a2263adb13e22e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id bea518045d1f7b0c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id d4800e9883ec51c5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id d072b6af4a8c0537 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method mugi \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id a8e3e639dca6e3a9 · open full run detail →
	mugi	gpt-4.1-nano	0.3718	0.9787	0.3843	0.5095	0.3171	0.6673	0.6900	0.9527	0.6317	0.1144	0.4072	0.4770	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3254	0.8105	0.6611	0.8904	0.6432	0.9203
methodmugi llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 5c480b8b1c025614 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id f3ef8c28a5c36e35 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 4788c1440c7ebced · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id c4f88c723d5a1b29 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 9ca313c0b69b3eea · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 01a43e6d888af2d7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 3b9d6b6773b91b59 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 9ed888a3da14d919 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method mugi \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 64efba2a45014195 · open full run detail →
	qa_expand	Qwen2.5-72B-Instruct	0.5174	0.9794	0.3830	0.5213	0.3333	0.6464	0.6796	0.9393	0.6324	0.1079	0.4168	0.4803	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3347	0.8285	0.6757	0.9005	0.6983	0.9284
methodqa_expand llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 158f4d081331a778 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 97e1f030ff0d3699 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 75929acc64d173b8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 4476b77e7af895c9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 47b49e007d29cc1b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 2af9d9ccf528d1d9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 326588da5cddb65c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id f862bc4b68efd409 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method qa_expand \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 438bf85c0461670d · open full run detail →
	qa_expand	Qwen2.5-7B-Instruct	0.5170	0.9829	0.3613	0.5111	0.2978	0.6387	0.6616	0.9547	0.6431	0.1103	0.3910	0.4548	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3143	0.8305	0.6574	0.8890	0.6156	0.8945
methodqa_expand llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 86132c2ba06bce75 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 3a815069c7c06d46 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 798e6395318b8632 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 4f85ddf666162891 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 319d5e8617b6f81f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id bd558adf58708c17 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id bc04280bf42f1888 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id e7cb3389f68a82b0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method qa_expand \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 5e53dafc053c231b · open full run detail →
	qa_expand	gpt-4.1	0.3823	0.9801	0.3873	0.5289	0.3399	0.6821	0.6964	0.9493	0.6941	0.1152	0.4266	0.4566	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3552	0.8034	0.7335	0.9170	0.6739	0.9260
methodqa_expand llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id b928c008fc588900 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id ecdcae2a6a8e4eb4 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 51feec618fc5f94f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id f43ee895ee17928d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 2a6b854819defbe0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 2b47882554623fb1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id c4fe891fb956c8d1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 982ea8c8e99ce5ea · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method qa_expand \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id dde02b83d639d757 · open full run detail →
	qa_expand	gpt-4.1-nano	0.3811	0.9787	0.4019	0.5396	0.3360	0.6669	0.6939	0.9420	0.7079	0.1215	0.4227	0.4696	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3702	0.8506	0.6883	0.9010	0.6628	0.9279
methodqa_expand llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 940d265efc31581a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id bc69f76b684cccf3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 9544e547f56b6e35 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 4e0076e1e487df2b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 949162b32988b17e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 58f6b4cbe43aa672 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id ba7b3397999a8c70 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id d7bf0033504f2795 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method qa_expand \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 741f8800fc2e11f3 · open full run detail →
	Q2D (FS)	Qwen2.5-72B-Instruct	0.5200	0.9801	0.3662	0.5023	0.3261	0.6552	0.7035	0.9567	0.6689	0.1142	0.4238	0.4846	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3333	0.8206	0.7151	0.9124	0.6499	0.9234
methodQ2D (FS) llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id d4c96038b2421b72 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 1431b663f3476c4a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 3b87caa9466242e1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 86a415d70ffeb20b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id b2bfdd905ff25123 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id f938ffe59e55de6a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id bfd75142f402414e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id ec284b56cc4b0cee · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 22abf0511cc254f7 · open full run detail →
	Q2D (ZS)	Qwen2.5-72B-Instruct	0.5194	0.9808	0.3707	0.5051	0.3213	0.6469	0.6965	0.9560	0.6272	0.1095	0.4068	0.4700	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3200	0.8248	0.6682	0.9161	0.6144	0.9161
methodQ2D (ZS) llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id e50ed88a6a0d9743 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id b2dbfb16c1c2a469 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 233ebdf8614f845f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 5302e9c1df55f57a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 17e0321bb5b85351 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id f5faf7ec25250d6b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id c6cd57f5e3a52bcc · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 92c02ea9d1094686 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 7f5e197e86320128 · open full run detail →
	Q2D (COT)	Qwen2.5-72B-Instruct	0.5199	0.9808	0.3897	0.5470	0.3157	0.6411	0.6834	0.9533	0.6425	0.1159	0.4054	0.4627	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3016	0.8393	0.6941	0.9148	0.6099	0.8857
methodQ2D (COT) llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 644866e8f19d2fc8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 51b13cae88a26722 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id b32f12da4795d84a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id f1a394c7e06f8f5b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 973fa6f6c5c68d50 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id d9f19520c34daa46 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id b3cf3dc3d5214a0b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 0190c9dc52a6cfc7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 33c4710d0519694c · open full run detail →
	Q2D (COT)	Qwen2.5-7B-Instruct	0.5200	0.9808	0.3697	0.5223	0.3206	0.6505	0.6825	0.9467	0.6567	0.1147	0.3831	0.4524	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2731	0.8239	0.6513	0.9037	0.5948	0.9019
methodQ2D (COT) llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 6fc0eb4976b64f71 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 8e1ae4b47cc703f1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 1ed323f21524e47a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 78db671246340f4a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 1c4a65da2e28111a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 0a9125d7fce9507b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id db5867912a6cff30 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 3038d21e605c99c3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 81ff403926fbda15 · open full run detail →
	Q2D (ZS)	Qwen2.5-7B-Instruct	0.5196	0.9815	0.3531	0.4926	0.3117	0.6509	0.6803	0.9567	0.6673	0.1124	0.4027	0.4812	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2904	0.8006	0.6091	0.8665	0.6096	0.9045
methodQ2D (ZS) llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 10750a2c759310d5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 2a4aea18fc844873 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id a2160aebb99d75bf · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 6b0354dd1f57f0fb · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 436633fd6e61404f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"zs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 83e36bbdcd01c854 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 4be12e26d83028c1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id db822818b660f509 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id b0a7a76ed835dd55 · open full run detail →
	Q2D (FS)	Qwen2.5-7B-Instruct	0.5199	0.9808	0.3575	0.4927	0.3079	0.6483	0.7120	0.9500	0.6793	0.1095	0.4146	0.4917	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.2672	0.8116	0.6095	0.8612	0.5492	0.9062
methodQ2D (FS) llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id e5853bd4e50372e7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 6781ed547701eeb3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 659853e299ae7307 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 0d861d7c7a402734 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 2b9c9a7ef6f17d79 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 320f704b7fecc186 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 7a85d826f607f964 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 3ce9480abb45ec9c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 57c4d760aeeba257 · open full run detail →
	Q2D (ZS)	gpt-4.1	0.3819	0.9808	0.3947	0.5209	0.3301	0.6766	0.7035	0.9553	0.6340	0.1089	0.4517	0.4786	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3377	0.8389	0.7000	0.9142	0.6875	0.9372
methodQ2D (ZS) llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id ac53933d5f31b244 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 9e53b84260c5b28a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id dc43a1f3d557cba0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 49ad57f49e20ba9e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 43081d6ebde85975 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 3119063da909dd39 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id b59cfe986a6af650 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 35705d0c3273a35d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 5859f8ccc1af1da7 · open full run detail →
	Q2D (COT)	gpt-4.1	0.3820	0.9801	0.3926	0.5319	0.3154	0.6513	0.7120	0.9460	0.6858	0.1056	0.4160	0.4741	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3308	0.8456	0.6877	0.9153	0.6534	0.9089
methodQ2D (COT) llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 4351eb7e929d6a1b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 59e4acd5f3038a1c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id a0a74603a124a27d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id d6b7380cd60c441e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 0ceb023e385feb76 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 037c4700e3fdd8a9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 6e8db20ea3cac77e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 58523e15f1b9f844 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id f5936331ecc5b7b8 · open full run detail →
	Q2D (FS)	gpt-4.1	0.3826	0.9808	0.3910	0.5192	0.3446	0.6890	0.7093	0.9567	0.6591	0.1099	0.4302	0.5009	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3771	0.8396	0.6932	0.9068	0.6749	0.9389
methodQ2D (FS) llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 8794c0d811dd0620 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 88e70d512f12573c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id b5323edbfe15c86b · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id d846402d817b7919 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id eea8b97081893f7f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 6f4bbd67016eedf4 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 68f40b0c1ca7bb4d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 29cc3246515308c3 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id f31f40bb5ce4a185 · open full run detail →
	Q2D (COT)	gpt-4.1-nano	0.3820	0.9801	0.3962	0.5324	0.3131	0.6532	0.7065	0.9433	0.6809	0.1163	0.4053	0.4554	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3426	0.8390	0.6544	0.8954	0.6271	0.9167
methodQ2D (COT) llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 03fecc6edc87cb5e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 21a45ce9a9f47965 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id cf7ba546c0d23e11 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id adae675b937bb6dd · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 1a09dc0368e366d0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id b9b57b08a675ecb0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id ff79e6533cfa9042 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 42977eb4a56aa064 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"cot","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 7f92bcedb7411662 · open full run detail →
	Q2D (FS)	gpt-4.1-nano	0.3823	0.9801	0.3790	0.5204	0.3390	0.6636	0.7121	0.9400	0.6715	0.1182	0.4292	0.4573	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3533	0.8005	0.6318	0.8839	0.6471	0.9232
methodQ2D (FS) llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 8381004825e16937 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 81b43c387e9d258d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 666a7b55b1d8b00d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 56a8fc44296dddef · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 001286d1ee758db1 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 80f477d3cc16b166 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 0abb612e9107c2a8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id b84b7426ddae8d52 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id da4c9162a78e48e2 · open full run detail →
	Q2D (ZS)	gpt-4.1-nano	0.3819	0.9808	0.3849	0.5064	0.3335	0.6640	23.0000	0.9493	0.6645	0.1146	0.4055	0.4651	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3479	0.8092	0.6877	0.8916	0.6242	0.9219
methodQ2D (ZS) llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id c7a440bf77f842d5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 1f21b6794dbb13f2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 1d598bfb41040ce9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 5e651b293bffa080 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id a1955739ce474a87 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id bae3507a2c36c1f0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 76130b6c1b9c60d0 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 9f0c9fedb962e02e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2doc \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train","mode":"zs"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 66fb380cf7931bd9 · open full run detail →
	query2e	Qwen2.5-72B-Instruct	0.5188	0.9808	0.3755	0.5195	0.3036	0.6438	0.7049	0.9427	0.6196	0.1201	0.4076	0.4799	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3442	0.8328	0.6686	0.9104	0.6353	0.9286
methodquery2e llmQwen2.5-72B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 14378080d839b817 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id fec87705663d16f9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id a78907f0a308a03d · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id c6d2a5ca2088376a · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 8ad0e2adba9b580e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id a15ecec4d8254e3c · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 5daf7ba27c702c81 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id e3c0316a5f22c1b8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2e \ --model Qwen/Qwen2.5-72B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id da479f6027f0f77d · open full run detail →
	query2e	Qwen2.5-7B-Instruct	0.5193	0.9815	0.3386	0.5134	0.2912	0.6256	0.7115	0.9493	0.6080	0.1093	0.4073	0.4888	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3056	0.7882	0.5474	0.8734	0.5312	0.9001
methodquery2e llmQwen2.5-7B-Instruct retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id 5a5070df3c8219ba · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 8660ffe293b039ea · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id f687bca9ceff1815 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id ff1e7a1c379a91d9 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 5cbf1afc48359dee · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id 2583e6668c775333 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id 92c57b2d152bec43 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 3b099faf31aa36d5 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2e \ --model Qwen/Qwen2.5-7B-Instruct \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id c4fbf354c7648c0e · open full run detail →
	query2e	gpt-4.1	0.3818	0.9808	0.3936	0.5477	0.3282	0.6670	0.7187	0.9393	0.6869	0.1222	0.4206	0.4992	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3518	0.8380	0.6812	0.9302	0.6522	0.9252
methodquery2e llmgpt-4.1 retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id c6e7b2a151f02ed7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 7a6ee0f58891ea5e · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id 1f66038780a30f17 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id 8eef829267cb36de · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id 2232f91adc848ab2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id aab784facc4eecd2 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id f44399bf64e99048 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 6a877049459faa16 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2e \ --model openai/gpt-4.1 \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id d578bd2ed58ea7f3 · open full run detail →
	query2e	gpt-4.1-nano	0.3819	0.9808	0.3716	0.5295	0.3113	0.6493	0.7206	0.9387	0.6747	0.1214	0.4086	0.4906	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	—	0.3297	0.8143	0.6320	0.9104	0.6605	0.9142
methodquery2e llmgpt-4.1-nano retrieverSPLADE++ 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-arguana \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-arguana.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-arguana-test run.txt Run id efe5039937455db6 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-dbpedia-entity \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-dbpedia-entity.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-dbpedia-entity-test run.txt Run id 4e04bf2e4a259995 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-fiqa \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-fiqa.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-fiqa-test run.txt Run id de6c83d1aac7b7d8 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-scifact \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-scifact.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-scifact-test run.txt Run id dbc1cc05086c4f87 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-covid \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-covid.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-covid-test run.txt Run id daa4e176a869fc62 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset beir-v1.0.0-trec-news \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index beir-v1.0.0-trec-news.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@100 python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.100 \ beir-v1.0.0-trec-news-test run.txt Run id cb48ef9af82a4711 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.dlhard \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt Run id a089e9f3c22d3e0f · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2019 \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl19-passage run.txt Run id 36d2ffae8e05a7d7 · open full run detail → 1 reformulate querygym → reformulated_queries.tsv python examples/querygym_pyserini/pipeline.py \ --dataset msmarco-v1-passage.trecdl2020 \ --method query2e \ --model openai/gpt-4.1-nano \ --steps reformulate \ --temperature 1 \ --max-tokens 128 \ --method-params '{"num_examples":4,"train_split":"train"}' \ --output-dir outputs/reproduce 2 retrieve pyserini · SPLADE++ (learned_sparse) python -m pyserini.search.lucene \ --threads 16 --batch-size 128 \ --index msmarco-v1-passage.splade-pp-ed \ --topics outputs/reproduce/queries/reformulated_queries.tsv \ --encoder naver/splade-cocondenser-ensembledistil \ --output run.txt \ --hits 1000 --impact 3 evaluate trec_eval · nDCG@10 + R@1k python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \ dl20-passage run.txt Run id 62c890d0402d7875 · open full run detail →