Data Exfiltration
A security threat involving the unauthorized transfer of data from a system or organization to an external destination. It is a significant concern in AI model security, particularly in cases where a model is suspected of transmitting user data to external servers via tool calls.
Core Facts
Timeline (last 90 days)
An attack can exfiltrate data by combining multiple legitimate tool calls: what a read tool reads can steer a write tool, which then sends it out through a notification tool, all without any tool being abused, any credential being stolen, or the model being jailbroken.
A read tool plus a write tool plus a loop constitutes a data exfiltration path unless some mechanism interrupts it at the tool boundary.
Data exfiltration can be achieved via image tags: if a reply generated by the Agent contains an image tag, when the message is rendered or the client loads the image, the data is sent to the attacker with the URL request.
Data exfiltration refers to an Agent being manipulated during task execution to send sensitive information to an external endpoint controlled by an attacker.
Permissions that appear harmless when held individually by an agent (such as reading files and calling an HTTP library) can enable data exfiltration when combined, which is the tool-combination emergence problem.
All Facts (5)
Permissions that appear harmless when held individually by an agent (such as reading files and calling an HTTP library) can enable data exfiltration when combined, which is the tool-combination emergence problem.
65%UnverifiedAn attack can exfiltrate data by combining multiple legitimate tool calls: what a read tool reads can steer a write tool, which then sends it out through a notification tool, all without any tool being abused, any credential being stolen, or the model being jailbroken.
50%UnverifiedA read tool plus a write tool plus a loop constitutes a data exfiltration path unless some mechanism interrupts it at the tool boundary.
50%UnverifiedData exfiltration can be achieved via image tags: if a reply generated by the Agent contains an image tag, when the message is rendered or the client loads the image, the data is sent to the attacker with the URL request.
50%UnverifiedData exfiltration refers to an Agent being manipulated during task execution to send sensitive information to an external endpoint controlled by an attacker.
50%