VeriCUA: Redefining Verification for Computer-Using Agents Through Agentic Exploration

Computer-Using Agents (CUAs) are becoming a key technology for automating operations across desktop, mobile, and web environments. However, for these agents to complete tasks reliably, one core challenge persists: how do you verify that a task has actually been completed? Whether it's data cleaning, model evaluation, or reinforcement learning, accurate judgment of task completion status is essential. Traditional methods either rely on Vision-Language Models (VLMs) to score trajectories, or generate programs to check application states—but both paths are limited by partial observation of complex digital environments and struggle to support reliable completion judgments. The recently proposed VeriCUA framework attempts to systematically address this problem with an entirely new approach: treating verification itself as agentic exploration.

Limitations of Traditional Verification Methods

Common automated verification approaches fall into roughly two categories. One has a VLM act as a judge, determining whether a task is complete based on provided observation records; the other generates code programs that directly inspect an application's accessible state. These two methods appear complementary but actually share the same weakness: they can only see part of the digital environment.

VLM judges are limited by the observation data fed to them—if key evidence doesn't appear in the screenshots, the model has no way to judge. Generated programs rely on the accessible state exposed by the application, and often fail when encountering unreadable UI elements or cross-platform differences. More troublesome is that teams must customize verifiers separately for different platforms and access permissions, with engineering costs growing linearly with the number of platforms. This fragmented approach is not only inefficient but also prone to misjudgment on difficult cases—either missing incomplete tasks or incorrectly accepting false completion claims.

Vision-Language Models (VLMs) are a class of multimodal large models that process both image and text inputs, with typical examples including GPT-4V and the Claude 3 series. In agent verification scenarios, VLMs are used as "judges": task descriptions and several frames of screenshots are fed in together, and the model outputs a judgment or score on whether the task is complete. The root of their limitation lies in the "observation window"—the model can only reason based on the limited frames fed to it. If key state changes occur between sampled frames, or are hidden in interfaces that require scrolling or page switching to see, the VLM cannot know. Additionally, VLM judgments are probabilistic, and subtle state differences (such as whether a toggle is truly on) are prone to hallucinated confirmation—this is also the fundamental reason why static judges struggle to support highly reliable verification.

Accessibility state refers to the interface structure information that an operating system or application exposes to external programs through accessibility interfaces, such as control trees, element attributes, text content, and selected states. Programmatic verifiers read this structured data to determine whether a task is complete—for example, checking the checked property of a checkbox. Its advantage is precision and assertability, but it depends on the application correctly implementing accessibility interfaces: custom-drawn interfaces, Canvas rendering, game engines, or cross-platform frameworks often cause missing elements or distorted attributes, at which point programmatic checks fail outright. This explains why programmatic verification performs so differently across platforms and applications, and constitutes the direct reason why platform-specific verifiers are needed.

VeriCUA's Core Idea: Verification as Exploration

The creators of VeriCUA redefine verification as "the process of dynamically collecting evidence and making assessments within a shared pipeline." Unlike static judgment, this framework has the verifier actively explore the environment rather than passively waiting for observation input. The entire process can be summarized in two stages: first, inspect the available context and build a task rubric; then, for unresolved requirements, choose between GUI inspection or code inspection, and synthesize findings from both under explicit evidence rules.

The key to this design is "agentic exploration"—the verifier is no longer confined to the initially provided observations but can actively interact with the interface to obtain additional information. For example, when a task requires confirming whether a setting has taken effect, the verifier can navigate to the relevant page and read the state through GUI inspection, or generate code to query the underlying configuration. The two evidence sources complement each other, reducing the blind spots that come from a single perspective.

More importantly, VeriCUA is a platform-agnostic framework—the same pipeline applies to both online and offline scenarios, covering different observation modalities. This means developers don't have to maintain separate verification logic for desktop, mobile, and web, significantly reducing engineering complexity.

A rubric in agent evaluation refers to a set of individually checkable completion conditions rather than a single holistic score. Building a rubric is the process of breaking down a vague task description into several specific, independently verifiable requirements—for example, "setting has been saved," "notifications are turned off," "file has been exported as CSV." This decomposition brings two benefits: first, each requirement can be matched with the most appropriate evidence-gathering method (GUI observation or code query); second, verification results are interpretable and failure points can be located. The quality of the rubric directly determines the ceiling of verification—omitted requirements cannot be recovered through subsequent exploration, so the first stage, "inspecting available context and building a rubric," is actually the most critical step in the entire process.

Test-time exploration borrows from the idea of test-time compute: investing additional computation at the inference stage rather than the training stage, improving result quality through multi-step interaction, trial and error, or search. In VeriCUA, this means the verifier can actively perform navigation, clicks, queries, and other actions before giving a judgment, trading increased environmental interaction for more thorough evidence. The contrasting paradigm is "one-shot judgment," where a single forward inference directly outputs a conclusion. The cost of test-time exploration is latency and operational risk (exploratory actions may change the state of the environment under test), so a trade-off must be made between evidence sufficiency and cost—which is also why "evidence rules" in the framework need explicit constraints.

Experimental Performance: Breakthroughs on Difficult Cases

According to the paper abstract, VeriCUA demonstrates higher consistency with reference labels across multiple experiments, outperforming the baseline methods compared. Two scenarios are particularly noteworthy: in offline evaluation, improvements on difficult cases are most pronounced; in online scenarios, the framework is stronger at rejecting false completion claims.

These two results point to the same conclusion: the complementary evidence brought by active exploration can indeed compensate for the shortcomings of partial observation. Difficult cases are often precisely those where key information is hidden outside the initial observation, while false completion claims exploit the blind spots of static verifiers. Through dynamic evidence collection, VeriCUA achieves better judgment in both types of challenging scenarios. Further analysis in the paper attributes the performance gains to complementary evidence and test-time exploration—that is, the verifier's ability to actively explore the environment during the test phase directly contributes to improved judgment accuracy.

Significance for the CUA Ecosystem

The value of VeriCUA lies not only in a verification framework itself. For teams building and scaling CUAs, reliable verification is infrastructure for data curation, evaluation, and reinforcement learning. If the verifier itself is unreliable, subsequent model training and product iteration will be built on erroneous signals. A platform-agnostic verification solution capable of active exploration means teams can obtain more trustworthy completion judgments at lower engineering cost, thereby accelerating the deployment of agents in real-world scenarios.

From a broader perspective, this research also suggests a trend: in complex digital environments, passive observation is often insufficient, and agents need the ability to actively acquire information—this applies not only to agents executing tasks but equally to agents verifying them. When verification itself becomes an exploratory behavior, the reliability boundary of the entire CUA system is expected to expand substantially.

For developers and researchers focused on deploying AI agents, VeriCUA offers a direction worth tracking: rather than customizing verifiers for each platform, design a general verification pipeline capable of adaptive exploration. This may become an important component of next-generation CUA infrastructure.

One-line Summary

VeriCUA reframes task verification as active agentic exploration, using complementary evidence to improve the reliability of completion judgments.

Paragraph Summary

The article introduces the VeriCUA framework, aimed at solving the core challenge of unreliable task completion verification for Computer-Using Agents (CUAs). Traditional approaches fall into two categories: VLM judges that score based on limited screenshots, and programmatic checks that read an application's accessible state. Both are limited by partial observation of digital environments and require platform-specific verifiers, resulting in high engineering costs and misjudgments on difficult cases. VeriCUA redefines verification as agentic exploration: first building a task rubric, then dynamically choosing between GUI inspection or code inspection for unresolved requirements, and synthesizing complementary findings under evidence rules. The framework is platform-agnostic, uniformly covering online and offline scenarios. Experiments show the most notable improvements on difficult cases, stronger rejection of false completion claims in online scenarios, with gains attributed to complementary evidence and test-time exploration.