[KongchangAI]
· 2 min read· 1,219 words

QDRAG: How Query Decomposition Improves RAG Citation Faithfulness and Generalization

QDRAG: How Query Decomposition Improves RAG Citation Faithfulness and Generalization

QDRAG improves attributed RAG citation precision in multi-hop multimodal QA via query decomposition and re-ranking pruning.

The article examines the performance degradation of attributed retrieval-augmented generation (RAGQ) in complex scenarios, noting that traditional RAGQ treats the entire question as the retrieval and citation unit, causing over-citation in multi-hop reasoning, failure to recognize implicit entities, and poor out-of-domain generalization. The paper proposes QDRAG, which decomposes questions into atomic sub-questions to make implicit entities explicit and uses a re-ranker to prune irrelevant contexts and reduce redundant citations. It offers two zero-shot decomposition methods, QD-C and QD-R, without extra training data, and reports a counterintuitive finding: fine-tuning on query decomposition generalizes better than fine-tuning the downstream RAGQ task. Experiments show QDRAG outperforms all baselines, including Gemini-Pro, on four multimodal QA benchmarks in both in-domain and out-of-domain tests.

Retrieval-augmented generation (RAG) has become a common approach to mitigating hallucinations in large language models, but traditional RAG often only provides answers without citing sources, reducing verifiability. Attributed RAG (RAGQ) improves credibility by generating citations that indicate which input contexts support the answer. However, existing RAGQ performs poorly on questions requiring multi-hop reasoning and multimodal understanding, facing three major pain points: over-citation, failure to recognize implicit entities, and poor generalization. A recent paper proposes the QDRAG framework, which systematically addresses these issues through query decomposition.

RAG citation generation diagram

Three Major Bottlenecks of Existing RAGQ

The paper points out that existing attributed RAG exhibits significant performance degradation in complex scenarios. Specifically, when questions involve multi-hop reasoning, models tend to over-cite from multiple irrelevant contexts, resulting in redundant and imprecise citations. Meanwhile, failure to recognize implicit entities (such as pronouns or elliptical expressions) causes retrieval and citation to deviate from the correct answer. In addition, the model's generalization ability on out-of-domain questions is insufficient, making it difficult to adapt to diverse multimodal question-answering scenarios. The root cause of these problems is that RAGQ treats the entire question as the unit of retrieval and citation, lacking fine-grained modeling of the question's internal structure.

Multi-hop reasoning refers to answering a question by chaining together multiple intermediate facts or reasoning steps, rather than directly extracting an answer from a single document snippet. For example, "Which film by a certain director won Best Picture?" requires first identifying the director's works, then verifying the award attribution. Traditional RAG's retrieval unit is an entire query, which in multi-hop scenarios tends to recall redundant documents covering multiple sub-facts, causing the model to mark irrelevant contexts as supporting evidence when generating citations, resulting in over-citation.

Implicit entities are entities that appear indirectly through pronouns, ellipsis, or anaphoric structures without an explicit noun phrase of their own, such as "the film he directed." Such expressions require contextual anaphora resolution to be converted into retrievable keywords; otherwise, retrieval deviates from the target entity, and citations fail accordingly. This is also an important reason for RAGQ's performance degradation in real conversational question-answering.

QDRAG: Decompose First, Then Retrieve, Then Cite

The core idea of QDRAG is to decompose the input question into atomic sub-questions to explicitly identify implicit entities, and then perform retrieval and citation generation on this basis. The framework includes two key steps: query decomposition and re-ranking pruning.

Query Decomposition: Atomic Sub-questions and Implicit Entity Recognition

QDRAG first uses a multimodal large language model (MLLM) to break the original question into several atomic sub-questions. Each sub-question corresponds to an independent entity or relation, thereby transforming implicit entities into explicit queries. For example, for a question like "What award did the film he directed win?", decomposition generates two sub-questions: "Which film did he direct?" and "What award did that film win?", ensuring that each step has a clear retrieval target.

To guide the MLLM through decomposition, the paper proposes two zero-shot methods: QD-C (context-knowledge-based) and QD-R (retrieval-reward-based). QD-C uses knowledge from the input context to guide decomposition, while QD-R optimizes the decomposition strategy through retrieval reward signals. Neither method requires additional training data, enabling effective query decomposition under zero-shot conditions.

Re-ranking is a two-stage paradigm in information retrieval: first, a recall stage (such as dense vector retrieval or BM25) quickly retrieves a large-scale set of candidate documents, and then a re-ranking model finely scores and reorders the relevance of candidate passages to the query. Unlike the recall stage, which pursues high recall, re-ranking focuses on high precision, typically using more computationally expensive models such as cross-encoders to evaluate query-passage matching pairwise. QDRAG uses this mechanism to perform secondary filtering on the candidate contexts for each decomposed sub-question, pruning passages with low relevance and suppressing redundant citations in subsequent citation generation at the source.

Re-ranking Pruning: Eliminating Over-citation

The decomposed sub-questions each retrieve relevant contexts, but the retrieval results may still contain distractors. QDRAG introduces a re-ranker to prune the contexts, removing passages irrelevant to the sub-questions and thereby reducing over-citation in downstream citation generation. This step ensures the precision of final citations, so that each citation directly supports the corresponding part of the answer.

An Interesting Finding: Fine-tuning on the Decomposition Task Outperforms Downstream Task Fine-tuning

The paper reveals a counterintuitive conclusion: fine-tuning on the query decomposition task exhibits better generalization ability than directly fine-tuning on the downstream RAGQ task. This means that learning the intermediate skill of "how to break down a question" helps the model adapt to new domains more than directly learning "how to generate citations." The authors believe this is because query decomposition forces the model to understand the semantic structure of questions rather than merely memorizing citation patterns, thereby achieving stronger transferability.

Zero-shot means that a model directly completes a target task without having seen labeled data for that task. The two decomposition guidance methods proposed by QDRAG, QD-C and QD-R, do not rely on additional training data: QD-C uses knowledge already present in the input context to prompt the MLLM to complete the decomposition, while QD-R uses retrieval effectiveness as a reward signal to select better decomposition schemes. This design allows the framework to be directly applied to new domains and modalities without reconstructing decomposition annotations for each dataset, thereby reducing deployment costs and partly explaining its robustness in out-of-domain tests.

Experimental Validation: Comprehensive Lead on Multimodal QA Benchmarks

QDRAG was evaluated on four multimodal question-answering benchmarks, covering both in-domain and out-of-domain tests. The results show that the framework significantly outperforms all baselines in answer grounding and citation faithfulness, even surpassing Gemini-Pro. Experiments demonstrate that QDRAG can not only effectively reduce over-citation but also improve the accuracy of implicit entity recognition and maintain robust performance in cross-domain scenarios.

Implications for RAG Development

The success of QDRAG suggests that improving RAG systems should not be limited to optimizing the retriever or generator; structured decomposition at the level of question understanding is equally critical. By breaking complex questions into atomic sub-questions, RAGQ can more precisely locate supporting evidence and generate more credible citations. In addition, the generalization advantage brought by fine-tuning on the decomposition task offers a new direction for improving RAG's adaptability in open domains. In the future, how to integrate query decomposition more deeply with multimodal retrieval may be a direction worth exploring.

One-line Summary

QDRAG improves the citation precision of attributed RAG in multi-hop multimodal QA through query decomposition and re-ranking pruning.

Paragraph Summary

The article focuses on the performance degradation of attributed retrieval-augmented generation (RAGQ) in complex scenarios, pointing out that traditional RAGQ directly treats the entire question as the unit of retrieval and citation, leading to three major bottlenecks: over-citation in multi-hop reasoning, failure to recognize implicit entities, and poor out-of-domain generalization. To address this, the paper proposes the QDRAG framework, which first decomposes questions into atomic sub-questions to make implicit entities explicit, and then uses a re-ranker to prune irrelevant contexts and reduce redundant citations. The decomposition step provides two zero-shot methods, QD-C and QD-R, without requiring additional training data. The article also introduces a counterintuitive finding: fine-tuning on the query decomposition task generalizes better than directly fine-tuning the downstream RAGQ task. Experiments show that QDRAG outperforms all baselines, including Gemini-Pro, in both in-domain and out-of-domain tests on four multimodal QA benchmarks.

#QDRAG#query decomposition#attributed RAG#RAG citation faithfulness#multimodal QA#re-ranking pruning#zero-shot
Share:

Related articles