AI Agents Are Testing the Limits of Zero Trust Security

When enterprises begin using orchestrators to manage a team of AI agent tasks—having them collaborate on risk assessment, customer profiling, matching, and information extraction—a seemingly basic yet extremely thorny question surfaces: Why should we trust that these agents actually executed our instructions? And where is the evidence?

This research article from Queen's University Belfast (QUB), published by IEEE Spectrum, brings Zero Trust Architecture (ZTA) into the discussion of agent governance. The core principle of zero trust is "never trust, always verify," but when the objects being verified become AI agents that act autonomously, invoke each other, and even migrate between tasks, implementing this principle is far more difficult than imagined.

Context Recovery Attacks: Deleted Data Can Still Be "Recalled"

The research team describes a typical risk scenario: to prevent agents from making biased decisions due to historical data, the system sets checkpoints and prompts agents to delete relevant historical context. But subsequent prompts may induce agents to "recall" information that should have been deleted, or even extract more content—this is the context recovery attack.

Such problems expose the core contradiction of zero trust in agent scenarios. ZTA relies on Policy Enforcement Points (PEPs) to maintain system trust, but the PEP itself must be trusted, so the question becomes: Who verifies the verifier? The article states bluntly that placing a single entity there and expecting it to scale to cover all agent interactions underestimates the problem space.

Another named risk is "vibe coding"—when using AI models to assist software development, handing over the entire codebase to untrusted entities outside the organization can cause vulnerabilities to spread across multiple layers.

A context window is the total amount of information a large language model can directly access in a single inference, including system prompts, conversation history, retrieved documents, and more. Since the model itself has no persistent memory, so-called "deletion" often just removes the corresponding text from the current context window, rather than completely erasing it from model parameters or external storage. Attackers can use carefully crafted follow-up prompts to exploit the model's tendency to complete context, inducing it to reconstruct or infer removed information based on residual clues. This mechanism creates an essential tension between "instruction following" and "information forgetting": the model is asked to forget a piece of content, but its generation process may still be influenced by statistical associations formed during training or prior interactions. Therefore, in agent systems, deletion instructions at the prompt level alone cannot constitute a verifiable guarantee of forgetting; external auditing and state isolation mechanisms must be introduced.

De-authentication Attacks Completed in 2 to 3 Seconds

Internal experiments at Queen's University Belfast provide an intuitive time scale: under agent orchestration, popular de-authentication attacks can be completed in as fast as 2 to 3 seconds. This means the time window left for the system to react and decide to patch is almost zero.

The article then introduces the trust challenge under asynchronous autonomy. Agents need to collaborate around shared goals—how do you ensure they are all aligned and understand the delays in their respective goals? The team's research path is to first prove the problem exists, then demonstrate how large the impact and scale can be when trust must be proven within asynchronous autonomous boundaries.

Their proposed mitigation directions include maintaining an AI bill of materials (AI-BOM), or using distributed ledger technology to provide trust infrastructure through traceability and immutability, making agent activities and their trust vectors platform-independent.

A de-authentication attack was originally a denial-of-service technique in wireless networks: an attacker forges management frames and sends de-authentication notifications to an access point or client, forcibly disconnecting legitimate connections. In agent orchestration scenarios, this attack surface is significantly amplified—agents frequently handshake and authorize through APIs, message queues, or shared state. Once the authentication state of a link is maliciously reset, the orchestrator may misjudge the node as disconnected and trigger reallocation or degradation policies, causing a chain reaction. The reason attacks can be compressed to seconds is that trust establishment between agents often relies on short-term tokens or session credentials, and automated orchestration logic responds extremely quickly to state changes, lacking a buffer for manual review. Defense approaches include rate-limiting critical authentication events, introducing out-of-band confirmation channels, and maintaining token revocation lists to block replay.

Horizontal Anomalies: Following the Rules in One Task Doesn't Mean Trustworthy Everywhere

The article specifically mentions a concept that has not been fully explored—horizontal anomalies. This refers to differences in agent behavior when interacting with different sources under different settings. An agent behaving correctly on a specific task or application and meeting architectural expectations does not mean its operations across all associated components can be trusted.

How to identify such anomalies and what methods can be used without complying with non-compliant agents remains an open question, the article admits.

Related to this is the cost trade-off. In systems with low fault tolerance and non-traceable agent workflows, real-time trust and verification of every query is computationally expensive, directly leading to the common trade-off between memory and execution. And once a zero-day attack occurs, over-reliance on an agent ecosystem with only a single trust layer may expose the entire ecosystem in the event of failure—just as there is no autonomy when the backbone network is unavailable, there is also no layer to place trust.

The AI Bill of Materials (AI-BOM) borrows from the Software Bill of Materials (SBOM) approach, aiming to establish a traceable component inventory for AI systems, covering model weights, training datasets, dependency libraries, fine-tuning configurations, inference framework versions, and external tools and APIs called by agents. In the context of agent governance, the value of AI-BOM lies in transforming "trust" from a static judgment of a single entity into dynamic traceability of the entire supply chain: when an agent behaves abnormally, one can locate layer by layer along the bill of materials whether it is caused by model version, data contamination, or tool interface tampering. Distributed ledger technology further provides this inventory with immutable timestamps and cross-organizational sharing capabilities, enabling trust vectors between different hosts to align. Combined, the two are expected to alleviate the dilemma of "who verifies the verifier"—the basis for verification is no longer the self-declaration of a centralized PEP, but independently verifiable traceability records.

Supply Chain Risk: One Agent Out of Control Is No Longer a Small Matter

The article's judgment on the cost of failure is worth heeding: agents under semi-trusted to low-trust governance bring short-term high returns, but the recovery cost when the system fails has not been explored, and it will not be a problem that can be contained.

For example, a supply chain problem caused simply by one agent going out of control is no longer a manageable small task, but requires sweeping all touchpoints of affected agents. This means that regardless of the number of agents or the ability to trigger proxy agents, it is necessary to deploy computationally expensive trackers for all agents.

The research team's hypothesis is that AI sovereignty and trust require collaborative governance, because traceability responsibility largely falls on the agent's host. There is a key distinction here: missing information and missing instructions are two different things, with fundamentally different implications for trusted accountability.

Does True Zero Trust Exist?

The article returns to the fundamental question: Does a true zero trust architecture exist?

The real crux is that these systems fail to adopt methods that historically relied on state modeling to govern trust. In stateless systems with high agent autonomy, the convergence and reliability of trust traceability significantly decrease. Can policy be trusted? If so, how should it be managed to avoid falling into the trap of policy-based access and control?

In the Cyber AI hub project at Queen's University Belfast, the team is collaborating with trust-first industries—organizations that treat trust as guardrails managed by internal policies, focusing on protecting systems through policy rather than trusting third-party agent service providers. Specific practices include setting constraints around service models, tracking and managing agent activities through logs, and maintaining token revocation lists to prevent replay attacks.

The article also mentions formal verification of trust: understanding model robustness helps build trust, and balancing adversarial accuracy with certified adversarial robustness can help determine the credibility of models. According to a research work cited by the team, adversarial distillation is used to improve certified robustness.

The question left for the industry is direct: When deploying agents in the real world, can we accept imperfect solutions? If so, how much impact can be controlled, and who should be responsible for the rest?

Key Takeaways

• De-authentication attacks under agent orchestration can be completed in 2 to 3 seconds, leaving an extremely narrow defense window
• Context recovery attacks can extract historical information that was supposed to be deleted, exposing gaps in instruction-following verification
• A single policy enforcement point is difficult to scale to cover all agent interactions, highlighting the zero trust "who verifies the verifier" problem
• Horizontal anomalies show that correct behavior on a single task cannot be extrapolated to global trustworthiness
• Distributed ledgers, AI bills of materials, and formal verification are seen as possible paths to building trust traceability