Run detail

7a85d826f607f964

Dataset

msmarco-v1-passage.dlhard

Method

Q2D (FS)

Model

Qwen2.5-7B-Instruct

Retriever

SPLADE++ (learned_sparse)

params_hash

534732e6

Queries

Metrics

ndcg_cut_10	0.2672
recall_1000	0.8116

Reproduce this run

Three steps: (1) reformulate the queries with QueryGym's example pipeline, (2) run retrieval with Pyserini, (3) evaluate with trec_eval.

1. reformulate

python examples/querygym_pyserini/pipeline.py \
    --dataset msmarco-v1-passage.dlhard \
    --method query2doc \
    --model Qwen/Qwen2.5-7B-Instruct \
    --steps reformulate \
    --temperature 1 \
    --max-tokens 128 \
    --method-params '{"mode":"fs","num_examples":4,"train_split":"train"}' \
    --output-dir outputs/reproduce

2. retrieve (SPLADE++)

python -m pyserini.search.lucene \
  --threads 16 --batch-size 128 \
  --index msmarco-v1-passage.splade-pp-ed \
  --topics outputs/reproduce/queries/reformulated_queries.tsv \
  --encoder naver/splade-cocondenser-ensembledistil \
  --output run.txt \
  --hits 1000 --impact

3. evaluate

python -m pyserini.eval.trec_eval -c -m ndcg.cut.10 -m recall.1000 \
  /mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv run.txt

Artifacts

Config

config.json

{
  "method_params": {
    "mode": "fs",
    "num_examples": 4,
    "dataset_type": "msmarco",
    "collection_path": "/mnt/data/son/data/msmarco/collection.tsv",
    "train_queries_path": "/mnt/data/son/data/msmarco/queries.train.tsv",
    "train_qrels_path": "/mnt/data/son/data/msmarco/qrels.train.tsv",
    "train_split": "train"
  },
  "llm_config": {
    "temperature": 1,
    "max_tokens": 128
  },
  "dataset_config": {
    "topics": "/mnt/data/son/Thesis/t5/data/dlhard/neutral_queries.tsv",
    "index": "msmarco-v1-passage",
    "num_queries": 50
  },
  "retrieval": {
    "retriever_id": "splade-pp",
    "paradigm": "learned_sparse",
    "params": {
      "model": "naver/splade-cocondenser-ensembledistil"
    }
  }
}