Q2D-Web

The Q2D-Web dataset contains a corpus of 190 million (190M) web documents.

This article is organized based on a brief disclosure from a single source; the complete construction methodology, specific benchmark results, and data availability of Q2D-Web remain to be confirmed by further official materials.

Using an agent to automatically complete query reformulation enables the scalable generation of nearly 70,000 queries while maintaining the semantic quality of the queries.

Q2D-Web's combined set has an average of 99.6 positive relevance judgments per query.

The Q2D-Web dataset contains 69,721 agent-reformulated queries.

Agent citations refer to the web content that an AI agent actively selects and cites when completing a retrieval task, and are regarded as an implicit relevance signal.

Q2D-Web introduces three mutually independent sets of relevance judgments to cross-validate the quality of retrieval results.

Q2D-Web's three relevance sets are agent citations, production web rankings, and a combined set expanded through LLM-based judgments.

One of Q2D-Web's core motivations is to reduce false negatives.

In Q2D-Web, a document will not be readily judged as irrelevant as long as it is deemed relevant by any one of the reliable signals.

One research purpose that Q2D-Web serves is to examine how differences in relevance definitions themselves affect model performance.

In Q2D-Web, LLM judgments are designed as one of three sets rather than the sole basis, in order to guard against the biases that LLM judgments themselves may introduce.

For teams developing retrieval or RAG systems, model evaluation results should not be tied to a single annotation framework; cross-validation from multiple perspectives is needed to obtain more reliable performance assessments.

Q2D-Web's data had personally identifiable information stripped out during the collection phase.

Q2D-Web's data is sourced from real production search behavior, rather than manually constructed or model-generated queries.

In Q2D-Web, each query is evaluated independently and has its own relevance judgments.

Evaluating the primary query and support queries separately decomposes the agent's query planning capability into quantifiable sub-tasks, making it easier to identify whether failures stem from intent misunderstanding or insufficient information completion strategies.

Details about the specific construction process, annotation standards, and accompanying baseline model results of the Q2D-Web dataset have not been fully elaborated in publicly available materials.

The Q2D-Web dataset was collected continuously over a period of nine months.

The Q2D-Web dataset covers ten languages and dozens of domains.