RAG Retrieval Layer Stress Test: Query-Targeted Poisoning Can Cause nDCG to Plunge by 60%

The security boundary of Retrieval-Augmented Generation (RAG) systems is often implicitly built on the assumption that "the retriever is only responsible for finding relevant documents." But a stress test targeting the retrieval admission stage shows that as long as an attacker can insert a small number of query-relevant documents into the corpus, the ranking quality of classic retrievers will suffer a precipitous decline. This study covers the SciFact and NFCorpus datasets, with a total of 623 BEIR queries, and systematically examines the impact of four variables: document content, attacker query information, poisoning budget per target, and contamination scope.

How Poisoning Breaches the Retrieval Admission Stage

External text in RAG only affects generation results after it is admitted by the retriever into the model context. The researchers applied pressure at this "admission" step rather than at the downstream instruction-following stage—in other words, they measured failure at the retrieval layer, not whether the model was ultimately induced to output incorrect content.

The key variables in the experimental design include: the content form of injected documents (full-text anchoring or appended text), the amount of query information the attacker possesses, the poisoning budget P for a single target, and the scope of targets covered by contamination. This multi-variable decomposition allows the contributions of different attack paths to be separated, rather than vaguely reporting that "poisoning works."

nDCG@10 is one of the most commonly used ranking quality metrics in information retrieval, short for Normalized Discounted Cumulative Gain. It measures how close the top 10 results returned by a retriever are to an ideal ranking: each result receives a discounted weight by position (higher for earlier positions), then relevance-graded values are accumulated, and finally the score is normalized by the ideal ranking score so that different queries can be compared horizontally. nDCG@10 ranges from 0 to 1, with higher values indicating that the top results are closer to the ideal state of "ranking the most relevant documents first." The 40.2% to 61.5% declines in this article refer to the relative decrease of this metric compared with the clean-corpus baseline, not absolute score values.

Five Documents Pull Classic Retrievers Down by 40% to 60%

In the isolated-target stress test, only five query-targeted documents per query were needed to reduce the nDCG@10 of classic retrievers by 40.2% to 61.5%. This magnitude means that even if the vast majority of content in the corpus is clean, as long as the attacker precisely injects a very small number of documents, ranking quality will substantially collapse.

Different retrievers are sensitive to different poisoning forms. For BM25, most performance degradation can be explained by "full-query anchoring"—injected documents fully cover the query terms, thereby obtaining high scores at the term-matching level. Appended text, by contrast, simultaneously perturbs LSA and Hybrid ranking, indicating that semantic-level ranking can also be manipulated, not just vulnerabilities in keyword matching.

BM25 is a representative classic sparse retrieval algorithm based on term matching and probabilistic ranking models: it saturates the frequency of query terms in documents, weights by the inverse document frequency of terms, and normalizes by document length. LSA (Latent Semantic Analysis), by contrast, maps term-document co-occurrence into a low-dimensional semantic space through matrix decomposition, capturing synonymy and co-occurrence relationships; it belongs to early semantic ranking approaches. Hybrid retrieval fuses sparse term-matching signals with dense/semantic signals for ranking. The three differ in their sensitivity to poisoning precisely because they rely on different sources of evidence: BM25 depends on literal term coverage, while LSA and Hybrid also depend on proximity relationships in semantic space, so "appended text" can perturb the latter two simultaneously.

Dense Models: Query Anchoring Remains the Dominant Factor

Pretrained BGE-small exhibited different exposure characteristics in validation. Under the setting using only query information with P=5, the target exposure rate reached 100% on both datasets; after adding injection payloads, SciFact's nDCG changed only from 0.269 to 0.271, while NFCorpus's exposure rate decreased slightly to 98.8%.

This set of data points to a noteworthy conclusion: for this dense model, query anchoring remains the dominant factor on the retrieval side, and the injection payload itself has limited impact on ranking metrics. The near-saturated exposure rate indicates that dense retrievers likewise lack effective admission filtering when facing query-targeted documents, but their failure mechanism does not completely overlap with BM25's term-matching vulnerability.

BGE-small is a lightweight version in the BAAI General Embedding model family (BGE series), belonging to dual-tower dense retrievers: queries and documents are separately mapped by encoders into fixed-length vectors, then ranked by cosine similarity or inner product. Such models do not rely on literal term overlap and should in theory be more robust to keyword-stuffing-style "full-query anchoring," but because their training objective is to bring semantically related query-document pairs closer, an attacker only needs to make injected documents semantically close to the target query to still squeeze into the top ranks. Target exposure rate refers to the proportion of target queries for which injected documents successfully enter the top retrieval results (usually the top k) under a given contamination budget; it is a direct indicator of whether admission filtering has failed.

Partial-Information Attacks Under Long Queries

Query length significantly changes the interpretation of partial-information attacks. On 162 SciFact long queries, just two random terms covered 19.9% of eligible terms and still produced a 39.5% Hybrid target exposure rate.

This shows that an attacker does not need the complete query to carry out effective poisoning—in long-query scenarios, a small number of random terms is enough to leverage a considerable exposure proportion. For defenders, this means risk assessments based on "whether the attacker knows the complete query" are overly optimistic.

Multi-Target Contamination and the Limits of Defenses

When the contamination scope expands to multiple targets, cross-target interference occurs, meaning that injected documents targeting one target affect the retrieval performance of other targets. This effect makes the actual impact surface of attacks more complex than single-target experiments suggest.

The study also examined two types of defense approaches. Overlap-based admission filtering can remove full-query poisoning documents, but the cost may be a sharp decline in clean retrieval utility—the trade-off between defense benefit and usability is very acute. Maximum Marginal Relevance (MMR), meanwhile, failed to change the exposure rate under the tested configurations, indicating that such diversity-oriented reranking strategies do not inherently possess anti-poisoning capability.

Maximum Marginal Relevance (MMR) is a classic reranking strategy that introduces a similarity penalty between candidate documents and already-selected documents in addition to relevance scores, thereby balancing relevance and diversity in the result list and often being used to mitigate information redundancy. It was not designed to counter poisoning: MMR only adjusts mutual redundancy among documents and cannot identify whether a document obtained inflated relevance by deliberately anchoring to the query, so it is unsurprising that the exposure rate remains unchanged under query-targeted poisoning. Overlap-based admission filtering, by contrast, detects the degree of term overlap between a document and the query before the document enters the index or context, intercepting documents with abnormally high overlap; the cost is that it may also wrongly penalize clean documents that are inherently highly relevant to the query.

Practical Implications of Retrieval-Layer Security

The value of this work lies in locating the RAG security problem to a specific, measurable stage. The 40.2% to 61.5% nDCG decline and the 100% target exposure rate both occur before model generation; once poisoned documents enter the context, whether downstream components are induced is no longer within the retriever's control.

For teams building RAG systems, this means corpus admission and source credibility verification need to be designed as an independent security layer, rather than hoping that reranking strategies or diversity mechanisms will solve it incidentally. How much clean retrieval utility a defense solution sacrifices while removing poisoned documents is an engineering decision that needs to be quantified in advance, not remedied after the fact.

One-line Summary

Five query-targeted documents can reduce classic retrievers' nDCG@10 by 40% to 60%; the retrieval admission layer is a key gap in RAG security.

Paragraph Summary

This study conducts a stress test on the retrieval admission stage of RAG systems, covering the SciFact and NFCorpus datasets and 623 BEIR queries, and systematically examines four variables: injected document form, attacker query information, single-target poisoning budget, and contamination scope. Results show that only five query-targeted documents per query reduce classic retrievers' nDCG@10 by 40.2% to 61.5%; BM25 degradation is mainly explained by full-query anchoring, while appended text also perturbs LSA and Hybrid ranking. The dense model BGE-small reaches a 100% target exposure rate under query anchoring, and injection payloads have limited impact on ranking metrics. In long-query scenarios, two random terms can cover 19.9% of eligible terms and produce a 39.5% Hybrid exposure rate. On defense, overlap-based admission filtering can remove full-query poisoning but may severely damage clean retrieval utility, and MMR fails to change the exposure rate.