The AI Engineer's Role Has Shifted: From Writing Code to Building Agent Factories

Introduction: Karpathy's Funnel and the Redefinition of the AI Engineer

At the AI Engineer New York conference, Arize AI co-founder Aparna Dhinakaran raised a pointed question: what do AI engineers actually do now? The question is worth revisiting because a tweet from Andrej Karpathy the weekend before the conference happened to capture the current state of affairs—over 6 billion people worldwide have interacted with LLMs, but the number of users actually running thousands of always-on agents at the frontier is probably only around 5,000.

Dhinakaran treated that tweet as a roadmap: over the next few years, how do we move more people from "chatting with an LLM" to "operating a team of agents"? Her answer is that the AI engineer's role is undergoing a fundamental shift—from writing features by hand to building the systems that let agents write features.

Always-on agents are agents that don't depend on a single conversation trigger but instead run continuously or periodically, autonomously perceiving their environment and taking action. Unlike chat-style LLMs, they typically have event- or cadence-based triggers, external memory, tool and repository permissions, and failure recovery mechanisms, allowing them to complete monitoring, diagnosis, and remediation tasks unattended over long periods. Karpathy's "funnel" refers to exactly this: billions of people have touched LLMs, but only about 5,000 users can actually operate thousands of such agents simultaneously. This gap reflects both engineering complexity and explains why the core work of AI engineers is shifting from writing features to building the loop systems that let agents run reliably.

From "Doing the Work" to "Building the Loop": The AI Engineer's Four-Level Leap

Dhinakaran sketched this shift with a set of clear contrasts:

• In the past, you wrote AI features; now you build the loop that writes features
• In the past, you read diffs for code review; now you build systems and agents to do the review
• In the past, you set up metrics and telemetry; now you let agents read observability data and find patterns
• In the past, you optimized manually; now you design a closed loop that optimizes itself

The loop she kept emphasizing is "build—evaluate—ship—monitor—improve." Traditionally, the AI engineer was responsible for executing each step of it; under the new definition, the engineer's core output is the loop itself. This view aligns with industry observations about the evolution of the "AI engineer" role: the level of abstraction keeps rising—from writing high-level languages to copy-pasting code, to calling local or cloud agents, and now the goal is running a team of agents.

Signal in Practice: How an Always-On Agent Operates

Signal, which Arize AI launched in its product, is the most concrete case in the talk. Signal is an always-on agent that reads all traces and telemetry data, identifies patterns, and—when it has repository permissions—directly submits fixes or PRs.

Triggering and the Sandbox Mechanism

Signal's startup can be configured: it can pull the latest batch of traces on a fixed cadence (say, every three to six hours), or it can be triggered by a Slack message, a user complaint, or a metrics anomaly event. Once awakened, the system builds a sandbox and places the trace context, third-party APM logs and metrics, repository access, and required secrets inside it. The sandbox has a single proxy egress, and everything inside is traced.

Sandbox is an isolated runtime environment for safely executing untrusted or high-risk code and operations. In agent scenarios, the sandbox encapsulates the context, tools, credentials, and network access the agent needs within a restricted boundary, keeping only a single proxy egress—this both prevents agent mistakes or malicious behavior from affecting production systems and makes it easy to fully record all actions into an auditable trace. Sandboxes are typically combined with a short-lived design: they are destroyed when the task ends, credentials and permissions expire with them, and a new instance is rebuilt on the next run. This pattern lets always-on agents repeatedly perform high-privilege operations like writing code and submitting PRs without accumulating long-term security exposure.

Memory and Durability Design

After each run, the sandbox is destroyed and credentials and repository permissions are removed, but discovered issues, the agent's own traces, and skills learned during the run are saved to external memory. On the next startup, the agent doesn't have to start from zero. Dhinakaran specifically mentioned "durability" design: if the sandbox or harness fails mid-way, the agent can recover from the failed step rather than rerunning the entire flow.

Model and Harness Agnosticism

Signal was initially built on Claude Code and now supports Cursor and Codex, with the goal of covering all mainstream and even open-source harnesses and supporting multiple models. This design serves two purposes: first, to be able to evaluate agent quality on every startup; second, to control cost—a stronger model may produce better PRs, but the team needs to verify whether issue-detection rigor holds up after downgrading.

Harness refers to the runtime framework that wraps the large model and drives the agent to complete tasks, handling orchestration logic such as tool calls, context management, loop control, and error recovery. Claude Code, Cursor, and Codex all belong to this category. Decoupling the model from the harness means the same agent logic can switch the underlying model or runtime without rewriting business code. This agnosticism brings three benefits: first, agent quality can be evaluated laterally across different combinations; second, stronger or cheaper models can be flexibly chosen based on task difficulty to control cost; third, it avoids lock-in to a single vendor, preserving replaceability and cost-reduction headroom in the infrastructure. The trade-off is the need for an additional adaptation layer and a unified observability interface.

The Evaluator Itself Also Needs to Be Evaluated

Dhinakaran believes AI engineers should spend more time writing evals. But a trickier question follows: how do teams trust the evaluators? Her answer is "evaluating the evaluators"—building golden-standard datasets, validating eval results against human outputs, and only shipping a new evaluator once the judge is confirmed to align with human judgment. These evals are used both for offline testing and for online evaluation in production.

This leads to her new definition of the AI engineer's core capability: defining "what good looks like." As agents take on more and more execution work, the value of human engineers increasingly concentrates on establishing and calibrating judgment criteria.

LLM-as-a-judge is an evaluation paradigm that uses a large model to score or judge agent outputs, commonly used for open-ended tasks that are hard to match with rules. Its core risk is that the judge itself may be biased, drift, or be unstable, which is why "evaluating the evaluator" is needed: first build a golden-standard dataset with human annotations, then measure the agreement rate between the judge and human judgment (e.g., accuracy, Cohen's kappa). Only after the agreement rate meets the bar is a new evaluator allowed to go live. Offline evals are used for pre-release regression testing, while online evals continuously sample production traffic for monitoring. Only by combining the two can the evaluation system itself become a trustworthy measurement tool rather than yet another black box.

Tracing Everything: From Agents to "Agents That Build Agents"

In the closing part of the talk, Dhinakaran pushed the scope of tracing to a lower level: not only should you trace the agents you build, but you should also trace the coding agents that build those agents—the call chains of Claude Code, Cursor, and Codex should all be recorded. Once these traces enter production, they become the data source for online evaluation, which then feeds back to the agents to discover new issues.

This forms a complete data flywheel: collect data, evaluate data, run agents on the data, discover issues, improve the system. She acknowledged that there is a clear gap between lab environments and real enterprises—enterprise data is far messier than the controlled, measurable data in labs—and the AI engineer is precisely the role that bridges this gap.

Editorial View: The Organizational Implications Behind the Role Definition

Dhinakaran's talk is essentially describing a shift toward "the AI engineer as systems architect." When the agent factory becomes the norm, engineers will no longer be measured by how many features they wrote, but by the degree of loop automation and the reliability of the evaluation system. This imposes new requirements on teams: someone needs to dedicate effort to maintaining golden datasets, observability infrastructure needs to support agent consumption rather than only human consumption, and teams need to accept the meta-work of "evaluators also need to be evaluated."

One detail worth noting is that Dhinakaran mentioned Signal's design principles came from "first-principles" thinking and deliberately kept the model and harness agnostic. This hints at a judgment: future agent infrastructure should be replaceable, comparable, and able to reduce costs, rather than locked to a single vendor. For enterprises currently building internal agent systems, this may be the most actionable advice.

One-line Summary

The AI engineer is shifting from writing features to building the loop systems that let agents write features.

Paragraph Summary

At the AI Engineer New York conference, Arize AI co-founder Aparna Dhinakaran used Karpathy's "funnel" tweet to point out that only about 5,000 people worldwide run thousands of always-on agents at the frontier, and that the AI engineer's responsibility is shifting from writing features by hand to building the "build—evaluate—ship—monitor—improve" loop itself. Using the product Signal as an example, she demonstrated the key design of always-on agents: event- or cadence-based triggers, single-proxy sandboxes, external memory and durability, and model/harness agnosticism. She also emphasized that evaluators themselves need to be evaluated, that the engineer's core capability is defining "what good looks like," and that tracing should extend to coding agents to form a data flywheel.