EvoX: Treating the Agent Harness as a Runtime Policy

EvoX treats the Agent Harness as a runtime operational policy, using three mechanisms to boost task success rates by about 3 percentage points.
The article introduces research on EvoX, which argues for repositioning the Agent Harness from engineering scaffolding to an "operational policy" complementary to the model's semantic policy. Built on Rust, EvoX uses three mechanisms—Action Realization, Execution Continuity, and Outcome Grounding—to address tool-call mismatches, workflow interruptions, and false completion claims, respectively. The study also proposes the Contrast evaluation framework, using fairness constraints, provider-bound request validation, and evidence-based adjudication to avoid evaluation interference with the Harness's native behavior. Experiments cover five benchmarks, six models, and six Harnesses, achieving the highest success rate of 56.2% across 691 task-model combinations, about 3.0 percentage points ahead of other Harnesses on average, with leads of 10.7 and 7.9 percentage points on GLM-5.3 and Kimi K3, respectively, showing model-dependent gains.
Harness Is More Than a Shell: From "Semantic Policy" to "Operational Policy"
Large language model (LLM) agents are increasingly interacting with external tools and environments, but plausible-sounding model decisions do not always translate into successful execution. Tool-call mismatches, broken workflows, and tasks the model claims to have completed but actually hasn't are common in real-world operation. A study from rss proposes EvoX, an attempt to rethink the role of the Agent Harness at the runtime level.

The core idea of EvoX is "The Second Policy": treating the Agent Harness as an Operational Policy that complements the base model's own Semantic Policy. In other words, the model is responsible for understanding intent and planning, while the Harness is responsible for reliably grounding that intent at the execution layer.
This distinction has practical significance. What the model outputs is "what it intends to do," whereas execution must handle details such as argument formats, tool interface constraints, call ordering, and state continuity. Built on Rust, EvoX is a multi-model agent runtime that combines model-facing guidance with deterministic runtime control to bridge the gap between semantic policy and execution outcomes.
Agent Harness refers to the entire runtime infrastructure wrapped around the base model that drives an agent to complete a task, typically including prompt orchestration, parsing and execution of tool calls, context and state management, error retries, and loop control. It determines how the model's text output is transformed into concrete function calls, file operations, or network requests. In early agent practice, the Harness was often viewed as "engineering scaffolding," with the focus on whether it could run at all; EvoX's argument is to treat the Harness itself as a researchable, designable policy layer—one that doesn't merely passively forward the model's intent, but actively constrains and corrects execution details. This shift in perspective means that the same base model paired with different Harnesses may yield significantly different task success rates, making the Harness a performance variable independent of the model weights.
Three Mechanisms: Making Decisions Actually Land
EvoX realizes the operational policy through three complementary mechanisms:
- Action Realization: maps the model's intent to actual calls that conform to tool constraints, reducing tool-call mismatches.
- Execution Continuity: maintains workflow progress under interruption and exception scenarios, preventing tasks from being abandoned midway.
- Outcome Grounding: supports completion claims with verifiable evidence, suppressing "claims completion but doesn't actually complete" behavior.
These three mechanisms correspond respectively to tool invocation, process coherence, and outcome verification, covering exactly the links in the agent execution chain that are most prone to distortion.
"Grounding" in Outcome Grounding is borrowed from the concept of grounding in NLP and knowledge representation, referring to anchoring abstract or symbolic statements to observable, verifiable real-world evidence. In agent scenarios, the model may generate a claim of "task completed," but that claim may not correspond to actual environment state changes. Outcome grounding requires that completion claims be accompanied by verifiable evidence, such as success status codes returned by tools, file system changes, test-pass records, or confirmation responses from external APIs. In the absence of evidence, the runtime can refuse to accept the claim and trigger remedial action. This mechanism targets the LLM's common "overconfidence" and "hallucinated completion," replacing natural-language self-reporting with objective judgments based on environment feedback, so that task completion status no longer depends on the model's self-assessment.
Contrast: An Evaluation Framework That Doesn't Suppress Native Behavior
Evaluating Harnesses has a difficulty: the evaluation method itself may interfere with the native behavior of the evaluated object. To address this, the study introduces Contrast, a controllable and auditable evaluation framework that includes explicit fairness constraints, provider-bound request validation, and evidence-based adjudication.
The purpose of this design is to let different Harnesses exhibit their own characteristics under relatively fair conditions, rather than being "disciplined" by the evaluation framework into the same behavioral pattern. This is especially critical for cross-model, cross-Harness comparisons.
The fairness challenge in Agent Harness evaluation is that if the evaluation framework imposes too many uniform rules on request formats, retry strategies, or tool sets, it will flatten the design differences among Harnesses and degrade the comparison into a measurement of adaptation to the evaluation framework. Contrast adopts "provider-bound request validation," maintaining the required request structure for each model provider to avoid misjudging engineering adaptation issues as capability deficiencies due to format incompatibility; at the same time, it replaces manual or heuristic scoring with evidence-based adjudication, reducing scorer subjectivity. This "controllable and auditable" design philosophy is similar to the concept of reproducible builds in software engineering—evaluation conclusions should be reproducible by third parties based on records, rather than relying on undisclosed proprietary processes.
Experimental Results: 56.2% Success Rate and Model-Dependent Effects
The experiments cover five benchmarks, six base models, and six Agent Harnesses. Across 691 shared task-model combinations, EvoX achieved the highest numerical success rate of 56.2%, about 3.0 percentage points above the average of the other five Harnesses.
More noteworthy is the differentiated magnitude of improvement: a 10.7 percentage point lead on GLM-5.3 and a 7.9 percentage point lead on Kimi K3. This shows that the effectiveness of Harness design is model-dependent, and also suggests potential for cross-provider adaptation. You may not have noticed that these results were achieved without using memory, evolution, or scaling mechanisms, indicating that the gains mainly come from operational control at the runtime level rather than additional capability stacking.
Significance and Limitations
The value of EvoX lies in elevating the Harness from "engineering scaffolding" to a research object of "runtime policy," and verifying its effectiveness with an auditable evaluation method. It reminds us that there remains a gap between the capability ceiling of base models and actual task success rates—a gap determined by runtime control.
At the same time, it should be noted that the average advantage of 3.0 percentage points is not overwhelming, and the gains vary significantly across models, indicating that Harness design needs to be adjusted to model characteristics. Whether EvoX can be extended to more tool ecosystems and longer-horizon tasks remains to be further verified.
Related articles

OpenAI Releases GPT-6.2 Dual Models Astra and Sol, API Price Cut by 20%
OpenAI releases GPT-6.2 Astra and Sol, with Astra's API price cut 20% to $8/$40 per million tokens and ranking second on the Artificial Analysis Intelligence Index. This article analyzes the dual-model strategy, price-cut impact, and developer selection advice.

Perplexity Computer Launches Multiagent: 16 AI Agents Cross-Validate Each Other
Perplexity launches Multiagent mode for Computer: a lead agent orchestrates up to 16 sub-agents on Claude, GPT, and Gemini, sharing findings and cross-validating work with effort above Ultra.

Amazon and Google to Jointly Hold Nearly 30% of Anthropic
After Anthropic's IPO, Amazon will hold just under 15% with an option to increase, and Google nearly 14%, together controlling close to 30%. This article analyzes the strategic logic, governance challenges, and impact on AI competition.