Prompt Injection Attack

Prompt注入（Prompt Injection）是针对大语言模型应用的安全威胁，攻击者通过在输入中嵌入恶意指令试图覆盖系统原有行为约束

Indirect prompt injection attacks plant hidden instructions within normal business data to induce Agents into executing unauthorized operations, representing a security risk at the semantic level rather than the code level.

Red-teaming originated in the military and cybersecurity fields and, in the AI safety context, has developed into multiple layers including prompt-level attacks, training infrastructure security testing, and model capability risk testing.

Prompt injection attacks do not need to exploit software vulnerabilities; they exploit the model's difficulty in distinguishing instructions from data.

Indirect injection refers to attackers hiding malicious instructions in external content that the model will read, such as webpage body text, email body text, or document content.

Because large models themselves cannot strictly distinguish between instructions and data, prompt injection is fundamentally very difficult to completely eliminate.

Large language models tend to solve problems using general reasoning while lacking strict adherence to domain-specific rules and self-verification capabilities.

Modern AI Agent frameworks require external tools to have deterministic inputs and outputs, predictable exit codes (0 for success, non-zero for failure), and machine-parsable output formats.

The core security risks of computer use capability include prompt injection attacks and unauthorized access to sensitive local data.

The core security risks faced by AI desktop agents include prompt injection attacks, privilege escalation risks, and irreversible operation risks.

Prompt injection attacks refer to malicious third parties embedding instructions in content processed by an Agent to hijack the Agent's intended behavior.

Prompt injection attacks exploit the model's blurring of the boundary between instructions and data, and are difficult to completely block through model-layer filters.

OpenClaw's documentation warns that smaller or heavily quantized models are more vulnerable to prompt injection attacks and recommends large models for any agent that uses tools.

Prompt injection attacks exploit large language models' tendency to follow instructions, inducing them to ignore the original system instructions, so security mechanisms that rely on prompts carry the risk of being bypassed.

Customer messages should be treated as task data rather than as authorization to modify workflows; this is a direct defense against prompt injection attacks.

Approaches to countering prompt injection attacks include enforcing human approval for sensitive operations, limiting agent permission scopes, and sandboxing execution environments, but no perfect solution currently exists.

A prompt injection attack refers to a situation where maliciously crafted web pages in an agent's environment inject new instructions, causing the agent's behavior to deviate from the designer's intentions.

Prompt injection exploits not a vulnerability in the code parser, but the model's own insufficient awareness of the boundary between instructions and data

The example finding given in the original was that an attacker used the Arabic ي to bypass the input filter, succeeding seven times out of ten attempts, with the model leaking the system prompt but not calling tools, rated medium severity; the root cause was that the filter compared raw text directly with no normalization at the boundary; after the fix, the same ten attempts yielded zero successes.