Prompt Injection
An attack technique targeting large language models in which adversarial instructions are embedded within external content read by the model, such as web pages, documents, or tool return values. This hijacks the model's behavioral logic without requiring direct interaction with the model.
Core Facts
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
Prompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
OWASP has listed prompt injection as the top security risk for large language model applications.
Timeline (last 90 days)
Prompt injection is ranked first on OWASP's list of AI risks.
Treating all read content as untrusted is the second line of defense against prompt injection, requiring agents to be designed with the ability to distinguish instructions from data, or to isolate external content in a "data zone" at the architectural level and prohibit it from triggering tool calls.
The team built a support Agent with deliberately planted vulnerabilities and demonstrated that a single line of instructions hidden in a ticket caused the Agent to pull customer account information and send it by email to the attacker's address.
Secure design needs to assume that the agent may be adversarial; even if current models are not malicious, future models or prompt injection attacks may cause agent behavior to deviate from expectations.
Prompt injection refers to an attacker hiding malicious instructions in content that the agent will read, causing the agent to mistakenly execute them as legitimate tasks, thereby making the assumption that "the current model is benign" no longer reliable.
Prompt injection exploits the inability of large language models to reliably distinguish instructions from data, and OWASP lists it as one of the top security threats for LLM applications.
In the new architecture, the sandbox no longer holds credentials; requests to external resources go through a proxy, which injects the token before calling external endpoints or MCP, and the model never sees the credentials.
Because LLMs cannot strictly distinguish 'instructions' from 'data' the way traditional programs do, prompt injection is difficult to eradicate with a single filter and usually requires a combination of defense-in-depth measures such as bidirectional input-output detection, least privilege, and tool-call whitelists.
srt addresses high-frequency risks such as privilege escalation caused by agent misoperation or prompt injection, not strongly adversarial multi-tenant isolation.
Prompt injection attacks in the MCP context can be carried out via a plain-text email: the model calls a read-only tool to read the email, the email contains malicious content such as "ignore previous instructions," and the model cannot always distinguish data from instructions, thereby attempting to perform dangerous operations (such as sending a private key file).
40 more timeline events
All Facts (20)
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
90%VerifiedPrompt injection attacks are a serious threat facing local AI Agents, as maliciously crafted content in files processed by the Agent may induce it to execute dangerous operations.
90%VerifiedOWASP has listed prompt injection as the top security risk for large language model applications.
90%VerifiedFailures in AI agents are often semantic deviations rather than explicit error codes, and this kind of "successful failure" requires the orchestration layer to have semantic-level error recognition capability.
85%VerifiedPrompt injection is divided into two categories—direct injection and indirect injection—where indirect injection refers to attackers concealing malicious instructions in external content that the AI may read, such as webpage body text, code comments, README files, and image OCR text.
85%VerifiedPrompt Injection is a security attack method targeting AI applications, analogous to SQL injection in traditional web development.
85%VerifiedPrompt engineering refers to the practice of carefully designing input text to guide AI models toward producing higher-quality outputs that better meet expectations.
80%VerifiedPrompt injection is essentially an architectural flaw in which large language models cannot distinguish data from instructions, analogous to traditional SQL injection in which the database engine confuses data with SQL instructions.
80%VerifiedBenchmark data contamination refers to the phenomenon where, after a test set is made public, its questions may appear in the pre-training corpora of subsequent models, causing scores to reflect memorization rather than genuine reasoning ability.
80%VerifiedIndirect prompt injection attacks plant hidden instructions within normal business data to induce Agents into executing unauthorized operations, representing a security risk at the semantic level rather than the code level.
80%VerifiedIn a Prompt Injection attack, an attacker can craft a special input to induce the Agent to perform unintended operations.
80%VerifiedOWASP已将Prompt注入列为LLM应用十大安全风险之首
80%VerifiedAI safety emphasizes defending against external threats and preventing system misuse, including prompt injection defense, jailbreak protection, data leakage prevention, and abuse detection.
80%VerifiedAI systems have introduced entirely new dimensions of security risk, including erroneous decisions caused by model hallucinations, prompt injection attacks, training data poisoning, and AI agent privilege abuse.
80%VerifiedPrompt injection refers to an attacker embedding malicious instructions into text that the model will read, causing the model to mistake these instructions for part of a legitimate task and execute them.
75%VerifiedLarge models lack a natural mechanism for distinguishing 'instruction source trustworthiness' when processing context, making it difficult to automatically identify which text constitutes authorized executable instructions and which is merely data content to be processed.
75%VerifiedIf the underlying data is messy, AI models will amplify those errors faster and at greater scale—meaning "garbage in, garbage out" is further magnified in the AI era.
75%VerifiedPrompt injection has become one of the most closely watched attack surfaces in the field of agent security
75%VerifiedThe concept of Prompt Injection was first systematically proposed by security researchers including Simon Willison in 2022.
75%VerifiedInternal agents that are maliciously prompt-injected or misled may execute dangerous operations while holding legitimate permissions.
75%