Running AI Agents for 16 Days Straight: Deconstructing Factory's Multi-Agent Architecture

Title

Running AI Agents for 16 Days Straight: Deconstructing Factory's Multi-Agent Architecture

Meta Description

Factory deconstructs the Missions multi-agent architecture: validation contracts define completion criteria before code, serial execution plus read-only parallelism lowers error rates, structured handoffs prevent context loss, and mixed models let planning, implementation, and validation each play to their strengths. In production cases, the longest continuous run was 16 days, with tests accounting for 50% of the final code.

Article Content

The bottleneck of a single AI Agent is often not that the model isn't smart enough, but that there aren't enough humans. Luke, head of Factory's core Agent team, made a judgment in his talk: the bottleneck of modern software engineering has shifted from intelligence level to human attention—engineers may have 50 tasks piled up, but can only push two or three forward each day, because every commit needs someone watching.

So what if humans are only responsible for deciding what to do, and an Agent system autonomously decides how to do it, running for several days straight, and when you come back the task is already done? This is exactly the starting point for Factory's Missions system. This system ran continuously for up to 16 days, and the team believes running for 30 days is not impossible.

Five Foundational Patterns of Multi-Agent Collaboration

Luke first offered a concise taxonomy for the chaotic multi-agent field, reducing frontier frameworks to five patterns:

• Delegation: One Agent dispatches sub-Agents to execute subtasks and collects results. It is the simplest form of multi-agent communication and the pattern most people implement first.
• Creator-Verifier: One side handles implementation, the other checks the work in a fresh context. The key is separation of responsibilities—the Agent that wrote the code has sunk-cost bias, while a verifier with clean context is more likely to spot problems. This aligns with the logic of human code review.
• Direct Communication: Agents talk directly without a central coordinator. State easily fragments, there is no single source of truth, and it is very hard to get right.
• Negotiation: Agents communicate around shared resources (the same API, the same codebase). The best scenario is a win-win positive-sum game.
• Broadcast: One-to-many transmission of state updates or global constraints. It may not look flashy, but it is crucial for coherence in long-running tasks.

Positive-sum game is a concept in game theory: participants can create value through cooperation that is greater than the sum of their individual actions, and both sides benefit, rather than one side's gain necessarily equaling the other's loss (zero-sum game). In the negotiation pattern, when multiple Agents need to reach agreement around shared resources (such as the same codebase or the same API quota), the best case is finding a solution that improves both sides—for example, one Agent gives up modification rights to a certain file in exchange for another Agent's compatibility commitment on the interface, and both ultimately complete their tasks faster. If they fall into zero-sum thinking, Agents will block each other and fight over resources, reducing overall efficiency.

Missions' Three-Role Architecture

Missions combines delegation, creator-verifier, broadcast, and negotiation into a single workflow: you describe the goal, clarify scope through dialogue, approve the plan, and the system executes autonomously for hours or even days. It consists of three roles:

Orchestrator is responsible for planning. Like a strategic advisor, it asks key questions, checks for blind spots in requirements, and ultimately produces a plan containing feature points, milestones, and a Validation Contract. The validation contract defines "what counts as done" before any code is written.

Worker is responsible for implementation. Each feature is assigned to a worker with a clean context—no accumulated baggage, no attention decay. The worker reads the spec, implements the feature, commits to Git, and lets the next worker inherit a clean workspace and a runnable codebase.

Validator is responsible for validation, and there are two kinds. The review validator runs test suites, type checks, and lint; the user testing validator is more like a QA engineer—it launches the application, interacts with the interface through computer use, fills out forms, clicks buttons, checks page rendering, and confirms that the feature flow works overall.

This validation path reveals Missions' most core design tradeoff: most of the wall-clock time in most tasks is spent waiting for real-world execution, not generating tokens.

Validation Contract: Countering "Tests Confirming Code Rather Than Validating Behavior"

Luke pointed out a common problem with coding Agents: the Agent writes the feature, then adds tests, all tests pass, coverage is maxed out, but those tests were shaped by the code rather than defined by the behavior the code was supposed to implement. Tests written afterward cannot catch bugs; they merely confirm existing decisions. Systems that rely on this kind of validation will drift sooner or later.

The validation contract is written during the planning phase, before code, and defines correctness independently of implementation. Contracts for complex projects can reach hundreds of assertions; each feature must satisfy one or more of them, and the union of all features must cover all assertions. After each milestone is completed, the two types of validators run in sequence, and neither has seen the code—validation is adversarial by design.

Validation Contract draws on the idea of Design by Contract in software engineering: before implementation begins, use testable assertions to specify the behavioral requirements the system should satisfy, rather than reverse-engineering tests after the code is written. It is equivalent to an "acceptance agreement" signed by both parties—the orchestrator represents the requirements side in defining "what counts as done," and workers and validators must deliver and check according to this standard. Unlike traditional requirements documents, the assertions in a validation contract must be automatically verifiable, for example "after submitting the form, a new record should be added to the database" or "after clicking the delete button, the item should no longer appear in the list." Because the contract exists independently of implementation, it naturally resists the bias of "tests shaped by code" and provides an objective basis for milestone checks.

Structured Handoffs: Don't Rely on Agent Memory, Rely on Mandatory Records

A system that runs for several days must solve context loss. After completing a feature, a Worker cannot just say "done"; it must fill out a structured handoff: what was completed, what was left behind, which commands were executed and their exit codes, what problems were found, and whether the procedures defined by the orchestrator were followed.

Errors are caught at milestone boundaries, corrective work is re-scoped, and the task self-repairs and gets back on track. This is not hoping the Agent remembers what happened, but forcing it to write it down and actually deal with the problem.

Prompt Caching is an optimization technique in large model inference: when multiple requests share the same prefix prompt (such as system instructions, long documents, or conversation history), the server can cache the computation results for that content, and subsequent requests only need to process the changed parts, significantly reducing latency and cost. In long-running systems like Missions, large amounts of text such as the orchestrator's plan, validation contract, and skill definitions appear repeatedly in every call to every Worker or Validator, and prompt caching can amortize these fixed overheads to a very low level. This is also why Factory can heavily use prompt caching in production cases—without it, the cost of multi-day tasks would balloon due to repeatedly processing the same context.

Why Serial Execution Instead of Parallel

Intuitively, ten Agents in parallel means ten times the throughput. But Factory tried it, and in the software engineering domain it doesn't work: Agents step on each other's changes, duplicate work, make inconsistent architectural decisions, coordination overhead eats up the speed gains, and tokens burn just the same.

Missions' approach is serial at the feature level, with only one Worker or Validator running at a time; read-only operations within a feature (searching the codebase, researching APIs, code review) are parallelized. On the surface it looks slower, but the error rate drops dramatically—for tasks lasting days, correctness compounds.

The accompanying Mission Control provides a dedicated view where you can see what the current Worker is doing, what problems the handoff summary reveals, how complete the project is, and how much budget has been consumed, allowing people to supervise asynchronously in the role of project manager, or simply go hang out with friends.

Mixed Models: The Right Model in Every Seat

The entire architecture has an implicit premise: use the right model for each role. Planning benefits from slow and deliberate reasoning, implementation benefits from fast code fluency and creativity, and validation benefits from precise instruction following. No single model or vendor can be best at all three.

Inside Factory, this new skill is called "droid whispering"—modeling in your head how different LLMs interact, where they fail, how failures accumulate over multi-day runs, and then consciously assigning model seats. Validation may even use a completely different vendor to avoid bias from the same training data. Model-agnostic architecture brings a structural advantage: if you lock into one vendor, you are constrained by the weakest capability in that family. Conversely, validation contracts and milestone checkpoints allow Missions to run quite successfully even with open-weight models.

Production Data: The Slack Clone Case

Factory demonstrated real performance with a Slack clone project: 60% of time and 60% of tokens were spent on implementation; validation almost never passed on the first try and always created follow-up features—which precisely proves the value of the QA loop; 50% of the final code lines were tests, with tests covering 90% of the code; prompt caching was heavily used to amortize the cost of long tasks.

In enterprise scenarios, Missions is used for overnight prototype validation, rapidly building internal tools, large-scale refactoring and migration, supporting ML research, and modernizing codebases so Agents can be more efficient within them. The capacity of a five-person team advancing 10 workflows simultaneously might increase to 30, allowing the team to focus on architecture and product decisions—and the codebase delivered is cleaner than when it started.

Defying the Bitter Lesson: Designing for the Next Model Release

Builders of multi-agent systems all fear that the next model release will make their architecture obsolete overnight. Missions' response is to put almost all orchestration logic into prompts and skills rather than hard-coded state machines—about 700 lines of text, where changing four sentences can significantly change execution strategy. Worker behavior is driven by skills defined by the orchestrator for each task; the only deterministic logic is thin and handles only bookkeeping: running validation and blocking progress when handoff issues remain unresolved. The system provides discipline, the model provides intelligence, using primitives the Agent is already familiar with, such as agents.md and skills.

This also echoes the talk's conclusion: human attention used to be the bottleneck, but now the economics are changing. Luke's actionable advice to the audience is direct—open OpenDroid, run slash missions, argue with the orchestrator about scope, approve the plan, and then go do something else.

One-line Summary

Factory launches the Missions multi-agent system, using validation contracts and structured handoffs to achieve multi-day autonomous operation, shifting the bottleneck from human attention to autonomous execution.

Paragraph Summary

Luke, head of Factory's core Agent team, points out that the bottleneck of modern software engineering has shifted from model intelligence to human attention, with engineers able to push only two or three tasks forward each day. To address this, Factory launched Missions, which combines delegation, creator-verifier, broadcast, and negotiation into a single workflow through a three-role architecture of orchestrator, worker, and validator. The system defines validation contracts before coding to prevent post-hoc tests from becoming mere confirmation of implementation; it adopts serial execution at the feature level and parallel read-only operations, using compounding correctness to counter error rates; and it enforces state recording through structured handoffs to achieve multi-day autonomous operation. In a production case, the Slack clone project spent 60% of time on implementation, with 50% of the final code being tests covering 90% of the code. The architecture writes orchestration logic into prompts rather than hard-coded state machines, allowing it to evolve with model upgrades.