Prompt Injection Is Not a Prompting Problem: The Missing Trust Boundary in RAG Pipelines

Retrieval-augmented generation (RAG) applications concatenate retrieved text with instructions and feed them into the model together. When that retrieved text is written by an attacker, the model receives attacker-supplied instructions and operator-supplied instructions through the same channel. From that moment on, no prompt wording constitutes a boundary—the attacker's text enjoys equal authority with its retrieval context. This is not a problem that can be solved by rewording prompts; it is a trust-boundary defect at the system design level.

The Missing Boundary

In traditional applications, data from untrusted sources is parsed, validated, and quoted—it does not become executable content. SQL injection became manageable because queries and data were separated, and that separation relies on mechanisms enforced by the database, not on developers remembering the rules.

Language models have no equivalent isolation by default. Their input is a token stream in which instructions and content are syntactically indistinguishable. Any text in the context window can influence model behavior; a document that deliberately embeds instructions and a document that merely happens to mention instructions are essentially indistinguishable to the model.

This root cause determines that placing security in the wording of prompts puts the boundary in the wrong place. What needs to be established is an enforcement mechanism akin to "separating queries from data," not relying on the model to "understand" which content is trustworthy.

The core of SQL injection defense lies in prepared statements and parameterized queries: the database driver processes the SQL statement structure and the user input data as two independent transmission and parsing passes, with the data portion strictly bound as a literal that is never parsed as SQL syntax. This isolation is enforced by the database engine and does not depend on whether the developer "remembers" to escape. Language models are entirely different—they have no syntactic mechanism for distinguishing instructions from data; all input is flattened into the same token sequence, and the model builds associations across the entire context window through attention, making it architecturally impossible to mark a piece of text as "data only, not executable." This means RAG pipelines cannot simply copy the parameterized-query approach and can only compensate for this architectural gap through external controls (permission constraints, output validation).

Where the Risk Is Exposed

The original article offers a key judgment: risk is proportional to what the application can do after the model outputs.

Read-Only Retrieval: Reversible but Serious

A retrieval system that answers questions over a corpus may leak content the user has access to or return incorrect information. This risk is serious but usually reversible—the impact stays at the information level and does not directly trigger external actions.

Tool Access: Irreversible Actions

The nature of systems with tool access is different. If the model can call APIs, send messages, write to databases, or execute code, then an executed injected instruction is a real action taken with the application's permissions and the user session's identity. That action may delete data, send emails, or invoke paid interfaces, and the consequences are often irreversible.

The gap between these two types of systems is a design decision, and that decision is often made implicitly in the convenience of "just adding a tool." Many teams consider only feature coverage when adding tools, without concurrently evaluating permission boundaries.

Controls That Reduce Exposure

The original article offers several control measures whose common logic is to shrink the consequences of a successful injection rather than to prevent injection from occurring.

Treat Retrieved Content as Data

Wrap retrieved content in structures the model has been trained to treat as untrusted, and exclude instructions from that structure. This helps but does not solve the problem—behavior under adversarial content is probabilistic. Any solution claiming "this makes injection impossible" deserves skepticism.

"Wrapping retrieved content in structures the model has been trained to treat as untrusted" usually means enclosing the retrieved text in special delimiter markers (such as XML tags, special tokens, or boundary markers declared in the system prompt) so that the model, according to its training distribution, tends to treat the enclosed content as reference material rather than instructions. But this method is essentially still a "probabilistic soft boundary": attackers can construct patterns similar to the delimiters within the wrapped content, exploit the model's comprehension biases regarding nested structures, or bypass patterns seen during training through rewriting. Multiple studies in the industry have shown that delimiter or system-prompt hardening alone sees a significant drop in protection rates against adaptive attacks. It can therefore only serve as one layer of defense in depth and cannot replace hard constraints such as the permission model.

Constrain Tool Permissions

Each tool should have the narrowest permissions that satisfy the use case, scoped to the current user rather than the application's service identity. Injected instructions can then cause damage only within the scope allowed by tool permissions. This is the most critical link in the entire protection system—the permission model is the only place where risk can actually be compressed.

"Service identity" refers to the pattern in which the application backend calls downstream APIs with its own credentials, where all users share the same set of high-privilege credentials; once the model is manipulated by injection, the attacker can act with the application's full permissions. "User scoping" requires tool calls to carry the current end user's identity token (such as an OAuth delegated token), so that downstream systems make authorization decisions based on that user's actual permissions. This is similar to row-level security in databases: even if the query statement is tampered with, the database still filters results according to the permissions of the current session principal. In practice, tool capabilities should also be minimized—for example, read-only tools should not be granted write permissions, email tools should be limited to a recipient whitelist, and database tools should be limited to specific tables and operation types—thereby compressing the reachable damage of a single injection to a business-acceptable range.

Require Confirmation for Consequential Actions

Actions that change state, send data, or incur costs should require human decision-making or an independent secondary check. This confirmation is a boundary the model cannot cross on its own. Handing irreversible operations to human confirmation essentially externalizes the trust boundary.

Validate Output by Schema

Validate before execution whether tool calls conform to the expected structure, and reject them if they do not match. This can catch garbled output and some redirection attempts. Such validation is low-cost with clear benefits and should be a default practice.

Restrict Runtime Egress

Retrieval pipelines rarely need arbitrary internet addresses. Restricting their network access can limit data exfiltration when the model is manipulated. This measure complements tool permission constraints: one governs "what can be done," the other governs "where data can be sent."

Detection Is Weak, So Design for Containment

The original article makes a pragmatic judgment: there is no reliable injection detector, because the carrier and the attack are the same channel. Filters can catch known patterns but fail in the face of rewriting.

Therefore, designs that rely on detecting injection are fragile, while designs that rely on limiting the impact of successful injection are durable. This aligns with the approach to memory corruption protection: you cannot prove that an exploit will never occur, so you limit the scope of damage and detect attempted behavior.

The focus of logging also needs to shift—record what the model did with tools, not just what it said. A tool call together with its parameters, the user session, and the documents it previously retrieved—these records answer the core questions of incident investigation and are often the only available evidence.

"The carrier and the attack are the same channel" means: the payload of an injection attack is natural language text, and the model's job is precisely to understand and follow natural language. Any detector must judge "is this text data or an instruction" within the same token stream, which is exactly the problem that the model architecture itself cannot reliably distinguish. Pattern-matching-based filters can only cover known attack strings, and attackers can bypass them through synonymous rewriting, encoding obfuscation, multilingual conversion, or fragment concatenation. This is similar to the reality in the memory safety field: Address Space Layout Randomization (ASLR) and stack canaries do not prevent all exploits, but they significantly raise the difficulty of exploitation and limit impact; logging and anomalous-behavior detection are used for post-hoc discovery and response. Migrating this mindset to RAG means shifting engineering investment from "preventing injection" to "limiting the damage radius after a successful injection" and "ensuring auditability."

Questions to Ask in Design Reviews

For every AI feature with tool access, the review should ask: if the retrieved content were written by an attacker, what is the worst action the model could take that the user would not notice?

That answer is the risk itself, and the tool permission model is the only place where it can be reduced. Embedding this question into the design review process is more valuable than any prompt hardening.

One-line Summary

The root cause of prompt injection is the lack of enforced isolation between instructions and data in RAG pipelines; the focus should shift to containment designs such as permission constraints.

Paragraph Summary

The article points out that prompt injection is not a matter of prompt wording but a missing trust boundary in RAG pipelines, where retrieved content and operator instructions share the same token channel. Language models cannot distinguish instructions from content the way databases separate queries from data, so placing security in prompts puts the boundary in the wrong place. The magnitude of risk depends on the executable capability after model output: read-only retrieval is reversible, while actions caused by tool access are often irreversible. Effective controls shrink the consequences of a successful injection rather than preventing injection, including structurally wrapping retrieved content, constraining tools to least privilege and user scope, requiring human confirmation for consequential actions, validating output schemas, and restricting network egress. Since detection is inherently unreliable, design should center on containment and ask in reviews what the worst unnoticed action an attacker could trigger is.