How Can Vertical AI Agents Differentiate? A Three-Layer Strategy and Regular Iteration
Vertical agents can differentiate via context, harness, and model—and must iterate regularly.
Based on a conversation with Harry Stebbings, the article proposes a three-layer differentiation strategy for vertical agents: context customization (prompts, skills, tools) is the easiest way to build a domain moat; the execution harness (planning, tool calls, error handling, memory management) can change the agent's fundamental behavior and is more difficult; model customization (fine-tuning or domain pre-training) is the hardest but most impactful. Since most engineers remain at the first two layers, establishing an advantage at the model layer first can yield a stronger moat. Given rapid base-model evolution, revisit each layer every 3–6 months and remove outdated parts (cruft).
In a recent conversation with Harry Stebbings, we explored many topics in the AI space, and the discussion about vertical vs. horizontal agents was particularly thought-provoking. I'm clearly bullish on the prospects of vertical agents and believe differentiation advantages can be built at three layers.
The Three-Layer Differentiation Strategy for Vertical Agents
1. Context: Quickly Build Domain Moats
Context includes prompts, skills, and tools, all tailored to a specific domain. Providing domain-specific context is relatively easy, yet it effectively helps the agent understand industry terminology, business processes, and user intent. For example, a legal-domain agent needs built-in statute retrieval, contract templates, and compliance-checking tools—these contexts form its first line of defense.
2. Harness: Adjusting the Agent's Fundamental Behavior
Compared with context, adjusting the execution harness is more difficult, but it can change the agent's fundamental behavioral patterns. The harness involves how the agent plans tasks, calls tools, handles errors, and manages memory. By optimizing the harness, you can make the agent work more efficiently in specific scenarios—for example, reducing unnecessary tool calls or improving the stability of multi-step reasoning.
Agent Harness is the runtime system that connects the large language model to the outside world, responsible for turning model outputs into concrete actions. It typically includes a task planner (decomposing complex goals into subtasks), tool-calling interfaces (executing API requests, running code, etc.), state and memory management (maintaining conversation history and intermediate results), and error handling and retry mechanisms. Unlike pure prompt engineering, the Harness determines the agent's behavioral boundaries and robustness in uncertain environments. For example, when a tool call fails, the Harness can decide whether to retry, degrade, or ask the user for clarification. Therefore, optimizing the Harness for a vertical domain (e.g., limiting concurrent tool calls, adding domain-specific validation steps) can significantly improve the agent's reliability and efficiency—which is also why customizing it is harder than customizing context.
3. Model: The Deepest Layer of Customization
Adjustments at the model layer are the most difficult, but they clearly change behavior the most. This includes fine-tuning base models or adopting domain-specific pre-trained models. Although costly, model customization can deliver significant performance gains for scenarios that require deep expertise.
Domain-specific pre-trained models refer to models obtained by continued pre-training on top of a general base model using large amounts of domain corpora (e.g., legal documents, medical records, financial reports). Unlike fine-tuning, continued pre-training not only adjusts task-related output layers but also updates all or most of the model's internal parameters, enabling the model to learn the language patterns, terminology distributions, and reasoning logic within the domain. This approach is costly and requires high-quality domain data and significant compute, but it brings deeper domain adaptation and lower inference costs (because lengthy prompts are no longer needed). For vertical agents, when context and Harness optimization hit a ceiling, domain-specific pre-trained models often become the key means to break through performance bottlenecks.
Most Engineers Are Still Stuck at the First Two Layers
From what I've observed, most agent engineers are still tuning context and the execution harness, and usually in that order. This makes sense: context is the easiest to start with, the harness comes next, and model customization has the highest barrier. But it also means that if you can establish an advantage at the model layer first, you may gain a stronger competitive moat.
Regular Iteration: Revisit Every 3–6 Months
Regardless of which layer you're tuning, given the rapid evolution of base models, it's advisable to revisit your context, harness, and model every 3–6 months. Upgrades to base models may render some old customizations redundant; promptly removing parts that are no longer needed (i.e., "cruft") keeps the agent lean and efficient. This habit of regular iteration is key to maintaining the competitiveness of vertical agents.
Conclusion
Differentiation for vertical agents isn't achieved overnight; it requires continuous investment across context, harness, and model, along with a mechanism for regular iteration. For founders, starting with context for rapid validation and gradually going deeper into the harness and model is a viable path.
One-line Summary
Vertical agents can differentiate across three layers—context, harness, and model—and require regular iteration.
Paragraph Summary
Based on a conversation with Harry Stebbings, the article proposes a three-layer differentiation strategy for vertical agents: context customization (prompts, skills, tools) is the easiest way to build a domain moat; the execution harness (planning, tool calls, error handling, memory management) can change the agent's fundamental behavior and is more difficult; model customization (fine-tuning or domain pre-training) is the hardest but has the most significant effect. The author observes that most engineers are still stuck at the first two layers, so establishing an advantage at the model layer first can yield a stronger competitive moat. Because base models evolve rapidly, it's advisable to revisit customizations at each layer every 3–6 months and remove outdated parts (cruft) to keep the agent lean and efficient. The conclusion notes that starting with context for validation and gradually going deeper into the harness and model is a viable path for founders.
Related articles
world2vec: Building Mechanistic World Models with Latent Dynamics Learning
world2vec proposes a reward-free method for learning mechanistic world models via latent dynamics, faithfully reproducing systems over long horizons with at least an order of magnitude fewer parameters than DreamerV3, TD-MPC2, and LeWM, and achieving accurate system identification on 71 attractor benchmarks.
A New KL-DRO Paradigm: Adaptive Scenario Generation Around a Learned Density Model
KL-DRO research proposes centering the KL ball on a learned density model instead of the empirical distribution, with adaptive scenario generation guiding sampling toward the worst case. Experiments show lower cost under distribution shift when records are scarce and the model can extrapolate, with comparable in-distribution cost.
ODE Samplers for Discrete Diffusion Models: Exponential Residual Flows Accelerate Generation
Discrete diffusion models lack deterministic ODE sampling? This article introduces Exponential Residual Flows (ERFs), which introduce continuous residual variables per token to construct piecewise ODEs for deterministic sampling and support consistency distillation acceleration. Superior to prior methods on code and text generation tasks.