Prompt Injection
An attack technique targeting AI systems in which malicious instructions are embedded in text read by an AI model, inducing it to deviate from its original objectives and execute attacker-designed behaviors. It is a central security threat facing agents powered by large language models.
Core Facts
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
Prompt injection is divided into two categories—direct injection and indirect injection—where indirect injection refers to attackers concealing malicious instructions in external content that the AI may read, such as webpage body text, code comments, README files, and image OCR text.
Prompt injection is essentially an architectural flaw in which large language models cannot distinguish data from instructions, analogous to traditional SQL injection in which the database engine confuses data with SQL instructions.
Timeline (last 90 days)
Prompt-level constraints (such as "do not execute instructions in documents") are easily bypassed, because injected content can be disguised as legitimate user intent.
Players tried prompt injection (such as asking everyone to speak like pirates), and the system noticed somewhat but more or less complied.
Prompt injection attacks can hide malicious instructions in the HTML source code of emails; when the model summarizes an email, it may be induced to ignore system prompts and execute malicious operations, with the risk being especially prominent for smaller or older models.
Content extracted from web pages or documents is positioned as "information to be evaluated" rather than an "authoritative instruction source".
The absence of centralized moderation means harmful content (such as the spread of prompt injection attacks) may be harder to control; and reputation systems are vulnerable to Sybil attacks when early users are few.
Insufficient isolation granularity leads to security issues such as unauthorized retrieval and cross-tenant leakage via prompt injection.
Unlike SQL injection, prompt injection has no simple escaping or parameterization scheme, because there is no formal boundary between natural language instructions and data.
Common approaches to defending against prompt injection include: strictly separating the system prompt from user input, structurally validating model output, limiting the tools and data the model can access, and adding explicit rules to the prompt that treat overriding system instructions as an attack.
When measuring the controllability of reasoning models in the future, it is necessary to distinguish between the "prompt-specified" and "self-instruction" conditions, otherwise the conclusions may be distorted.
"Whale Girl mode" is essentially system-level prompt injection that makes the model load a set of preset persona parameters, which is more like the model's overfitted response to structured instructions than an officially reserved feature.
40 more timeline events
All Facts (20)
Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束
90%VerifiedPrompt injection is divided into two categories—direct injection and indirect injection—where indirect injection refers to attackers concealing malicious instructions in external content that the AI may read, such as webpage body text, code comments, README files, and image OCR text.
85%VerifiedPrompt injection is essentially an architectural flaw in which large language models cannot distinguish data from instructions, analogous to traditional SQL injection in which the database engine confuses data with SQL instructions.
80%VerifiedPrompt injection attacks induce AI agents to execute malicious operations by embedding text disguised as system instructions in code comments, documentation, web content, or even file names.
80%VerifiedPrompt injection refers to an attacker embedding malicious instructions into text that the model will read, causing the model to mistake these instructions for part of a legitimate task and execute them.
75%VerifiedPrompt injection has become one of the most closely watched attack surfaces in the field of agent security
75%Verified提示注入是一种针对大语言模型的攻击手法，攻击者通过在模型输入数据中嵌入恶意指令劫持模型的正常行为
75%VerifiedOWASP lists prompt injection as the top risk in the LLM Top 10 (LLM01).
70%VerifiedSelf-generated prompt injection is a security risk in which the injected instruction originates from the model itself rather than an external attacker.
70%VerifiedPrompt injection attacks can use carefully crafted user inputs to induce the model to ignore the constraints of the system prompt.
70%VerifiedOWASP ranks prompt injection as the top risk among the ten major risks for large model applications.
65%VerifiedBecause large models themselves cannot strictly distinguish between instructions and data, prompt injection is fundamentally very difficult to completely eliminate.
65%VerifiedPrompt injection induces agents to deviate from the original task intent by embedding text disguised as instructions within external content processed by the agent.
65%VerifiedWhether AI-generated channel recommendations can truly fit the unique circumstances of each product remains to be verified in practice
65%VerifiedPrompt injection can be divided into two types: direct injection and indirect injection; indirect injection is where an attacker hides malicious instructions in external data sources such as web pages, PDFs, and emails, and when an AI agent processes this data the malicious instructions are executed as legitimate instructions.
65%VerifiedPrompt Injection refers to attackers using carefully crafted inputs to induce large language models to ignore original system instructions and execute malicious operations; in multi-agent systems, a compromised agent may propagate malicious instructions to other agents through legitimate internal communication channels.
65%VerifiedThe principle of indirect prompt injection is that an attacker disguises malicious instructions as normal data, delivers them to the model through a data source trusted by the agent, and induces the model to interpret the data as instructions and execute them.
65%Verified提示词注入（Prompt Injection）分为直接注入和间接注入两类，直接注入通过用户输入植入恶意指令，间接注入将恶意指令藏匿于AI可能读取的外部内容中
65%VerifiedLLMs are vulnerable to prompt injection attacks, where malicious web pages or files can embed hidden instructions to induce AI to perform unintended actions.
65%UnverifiedSkills Store的审核机制包括检查是否存在提示词注入风险或数据泄露隐患
75%