How to Safely Ship AI-Written Code? Breaking Down the Claude Code Team's Review Process

AI writes code faster and faster, but are you comfortable handing over code review and deployment entirely to AI? As the number of changes merged each day skyrockets, line-by-line human review can no longer keep up. Anthropic recently shared the Claude Code team's code review process, offering a reusable answer: with multi-agent collaboration, the number of merged PRs doubled while the proportion requiring rollback dropped by more than 40%. How does this process work? And what can ordinary teams learn from it?

Review Pressure from Exploding Code Output

In Anthropic's monorepo, daily merged changes grew from about 2,500 in May to over 30,000 in September. This order-of-magnitude shift shows that when code output grows rapidly, line-by-line human review faces enormous pressure.

To solve this, the Claude Code team introduced a Code Review process comprising verification, CI, AI review, security review, human approval, merging, and fast rollback. Let's walk through the whole flow using a concrete feature—changing the order amount calculation rules.

Development Environment Verification: Dual Self-Testing of Feature and Code

After building a feature, the first step is verification. This has two parts: first, verifying whether the feature meets requirements, which can be done in a local agent development environment. For example, if you built a feature with Claude Code, you need to provide the corresponding tools and environment to check the actual results—if it's a web page, let it be accessible via a browser; if it involves UI, provide screenshots as reference for it to check against.

Second is code review. The key practice here is: hand the requirements and code changes to another agent to check. Why use a new agent? Because the agent that wrote the code retains the prior reasoning and conversation context, and when it looks back it may follow the same line of thinking and miss the same problems. An agent with fresh context can provide an independent second opinion. You can also try having other models cross-check, though switching models doesn't necessarily mean better review, so many people use different models and different AI tools for cross code review to ensure quality.

This step is equivalent to self-testing before a developer submits code: verify the feature first, then check the code, and once done you can submit the PR.

CI Checks: The System Re-checks Automatically After Submission

After the PR is submitted, the project automatically runs a pre-configured set of checks—this is CI (continuous integration). Take order amount calculation again: you've already verified locally that item total minus discount plus shipping is correct, but could this change break other order scenarios? For example, how is it calculated when there's no coupon, or when multiple coupons stack? These scenarios can be written as tests in advance and run automatically after submission.

Beyond business tests, the system can also check whether the code compiles properly, whether there are common code issues, and whether the whole project packages successfully. What exactly gets checked is configured in advance by the project, and results appear on the PR page. If a required check fails, fix the code based on the error message and resubmit; only after checks pass can merging continue.

Continuous Integration (CI) is a development practice: developers frequently merge code changes into the mainline, and each merge triggers automated build and test processes to catch integration errors early. In the PR context, CI is typically triggered automatically by code hosting platforms (such as GitHub Actions, GitLab CI), running pre-configured scripts to perform compilation, unit tests, static code analysis, packaging, and other tasks. The value of CI lies in transforming "integration" from a one-time, high-risk manual operation into a continuous, small-step, repeatable automated process, significantly reducing the probability that problems are only discovered after multiple changes are combined.

AI Review: Multi-Agent Parallelism and Voting Mechanism

After the PR is submitted, besides running CI checks, it enters a very important stage—AI review. Multiple agents search for problems in the code in parallel. They first read the PR description to understand the intent of the changes, then review using the code changes and the context of the entire repository.

Review focuses on several aspects: whether this change solves the problem, whether it fits the existing system, and whether it affects other features; if it's a bug fix, it also checks whether the root cause was found. For example, when the order amount calculation rules are modified, AI not only checks whether this amount is calculated correctly, but also checks whether the calculation order of coupons and discounts conforms to existing rules, and whether modifying this shared method affects refund amount calculations.

Problems found by AI are not all handed directly to the developer. A group of agents verifies and votes on each issue, and only when enough agents believe the issue genuinely exists is the corresponding review comment posted. This process also applies filtering rules, for example comments involving only code style, formatting, or code cleanup are filtered out. The remaining review comments are sorted by severity and labeled as important issues, minor issues, or pre-existing issues—pre-existing doesn't mean it's not serious, but that it existed before this PR, avoiding pinning old problems on the current author.

Every discovered issue requires a response from the PR submitter: fix it if it's real; if it's believed to be a false positive or there's a reason not to change it, explain why and mark the issue as resolved. This process can also be assisted by agents. As long as there are unresolved findings in defect review or security review, the PR cannot be merged.

Avoiding Infinite Loops in AI Review

When using multi-agent review, a common problem arises: AI review raises issues, the coding AI fixes them and submits back for review, this round finds a few new issues, so it keeps fixing, and it may go back and forth dozens of rounds without ending.

Why does this happen? Because each check covers a large scope, the reviewing AI may keep finding issues in code it has already seen that it hadn't noticed before—these issues aren't necessarily caused by the latest fix, but as long as each round produces new comments, the fix-review cycle continues.

This can be controlled in three ways:

Set a round limit. If three rounds pass without approval, prompt the author to pause and reconsider the overall approach. For example, if modifying the order amount calculation, and after three consecutive rounds the AI still gets the stacked coupon and full-reduction results wrong, it's time to reconfirm what order the discounts should be calculated in and whether the implementation approach is flawed, rather than continuing to patch a wrong line of thinking. Note this doesn't mean auto-approving after three rounds, but that rethinking is needed.

Gradually narrow the re-review scope. In the first few rounds, check this change and its impact comprehensively against the whole project; in later rounds, focus on the newly added diff. For example, if the order calculation logic and related calls have already been reviewed, and this round only adjusted the amount and input method, the re-review focuses on this adjustment and its impact, reducing repeated checks of already-reviewed content.

Make comments clear in one pass. For example, if a precision risk is found from using double for an amount calculation, have the author check other similar amount issues together, avoiding fixing one spot this round and discovering another below. Issues already raised and addressed should also not be reposted every round, forcing the author to respond repeatedly. This way each review round has a clear scope, pushes issues toward resolution, and reduces the situation where two AIs keep modifying each other's work without ever finishing.

Multi-Agent System refers to an architecture in which multiple agents with autonomous decision-making capabilities collaborate to complete tasks. Each agent has an independent context, role, or goal, and results can be aggregated through parallelism, voting, debate, and other mechanisms. In code review, multi-agent parallel review can cover broader perspectives, while the voting mechanism is used to filter out false positives from individual agents and increase the confidence of review comments. The core advantage of this architecture is that a single agent's judgment may be limited by its context or randomness, while the consensus of multiple independent agents can significantly reduce the error rate.

Security Review: Covering Common Vulnerability Patterns

Next is security review, which checks code against common vulnerability patterns, focusing mainly on the following categories:

• SQL injection: User input is concatenated directly into database query statements, potentially allowing users to alter the original query logic through special input.
• Cross-site scripting (XSS): User-submitted content is displayed directly on web pages without proper handling, potentially allowing malicious scripts to execute in other users' browsers.
• Authentication and access control: For example, the order endpoint only checks whether the user is logged in but doesn't check whether the order belongs to that user, allowing others' orders to be viewed and modified.
• Insecure data handling: The backend doesn't validate input and directly trusts prices or discount amounts passed from the frontend, potentially allowing users to tamper with the order's payable amount.
• Known vulnerabilities in third-party dependencies: Frameworks and packages used by the project may also have security issues.

Teams can adjust check rules and sensitivity according to the project's security requirements. After issues are found, they need to be fixed, or confirmed as false positives with an explanation; if there are unresolved security findings, the PR cannot be merged.

SQL injection is a common injection attack: when an application concatenates user input directly into SQL query statements, an attacker can change the query semantics by crafting special input (such as ' OR '1'='1), thereby bypassing authentication and reading or tampering with database data. Cross-site scripting (XSS) is outputting unfiltered user content into web pages, causing malicious scripts to execute in other users' browsers, potentially stealing cookies or session information. These two types of vulnerabilities have long ranked high on the OWASP Top 10, and one of the core goals of security review is to identify patterns of "unvalidated user input entering sensitive contexts directly."

Human Approval: Risk-Tiered Participation

The core of the human approval stage is deciding how many people need to participate in review based on the risk of the change. Anthropic divides risk into three levels: low, medium, and high.

Low risk, such as documentation, copy, and features used only internally, can skip human approval after automated review and security checks pass. Medium risk, such as ordinary product feature changes and changes involving sensitive areas like cache auto-updaters or system prompts, requires one human reviewer's approval. High risk includes the highest-risk code with the highest level of human approvers, as well as a small number of code paths explicitly requiring two approvals; such changes require two human reviewers' approval.

Risk levels are not chosen arbitrarily by the person submitting the PR. Anthropic uses a classifier to determine risk based on the specific code changes, and another classifier to check whether the change requires additional human involvement.

Taking order amount calculation as an example, an error could cause users to overpay or merchants to undercharge, so such changes can be designated as high risk, requiring two reviewers.

Merge Queue and Post-Release Monitoring

Once the preceding checks and approvals meet requirements, the PR enters the merge queue. The queue adds the changes to be merged on top of the latest main branch and re-runs checks, merging officially only after passing; subsequent PRs are also checked against the updated main branch, reducing the risk of multiple changes each passing individually but failing when combined.

For example, one PR modifies the order amount calculation rules and another modifies coupon handling logic; their individual tests pass, but combined they may calculate amounts incorrectly. The merge queue exists to reduce such combination problems.

After merging, new features are usually placed behind feature flags and first opened to Anthropic's internal employees—who use Claude Code heavily every day—giving them a chance to find problems before ordinary users encounter them. External releases are also staged: Claude releases through two channels, with the Lattice channel for a small number of new users and the Step channel for the stable version. New versions enter the Lattice channel first, with the Step channel following, letting versions undergo a period of use and observation before gradually expanding.

Even after these checks, errors can still reach production, so continuous monitoring of runtime behavior is needed after release. If the problem involves a feature behind a flag, the flag can be turned off first to stop exposure; if the flag can't solve it, revert the problematic change as appropriate, or fix it and release a new version. The focus at this layer is to detect problems quickly and have a way to rapidly restore control over user impact.

Feature Flag is a mechanism that allows a feature to be dynamically enabled or disabled without redeploying code. When a new feature launches, it can first be kept off, then gradually opened to internal employees, a small number of users, or specific regions through configuration. Once a problem is found, the flag can be turned off immediately to stop the impact quickly without rolling back code. Combining feature flags with staged releases (such as canary releases and gray releases) can significantly reduce the risk of a new version to all users and buy time for troubleshooting.

Where Human Attention Should Go

When a large number of routine checks are handed to machines and AI, where should human attention go? This doesn't mean people are absent from PR review, but that based on risk and verification evidence, effort is spent where judgment is most needed. Using the order feature as an example, this can be divided into five aspects:

1. Deciding whether it's worth doing, and setting direction, scope, and user experience. For example, whether to build full-reduction, coupons, and points deduction all at once, or do one type of discount well first—AI can assist with analysis, but what to build and to what extent still needs human decisions.
2. Reviewing high-risk code. For example, after order amount changes, the discount calculation order, amount precision, and whether the order amount and payment amount are consistent are all worth focused human checks.
3. Governing architecture design. For example, order payment and refund all need to handle amount calculation; where the rules should live and which modules are responsible for calling them need to be thought through in advance, avoiding each module writing its own version.
4. Reading verification evidence. When AI says tests passed, look at which scenarios it tested—only ordinary orders, or also discount stacking, refunds, and other results—and judge whether the tests cover important risks.
5. Taking responsibility for outcomes. If users report incorrect amounts after launch, someone needs to assess the impact scope, decide whether to disable the feature, fix, or roll back, and follow up on affected orders. Although the code is written by AI, someone still needs to be accountable for the delivered result.

The value of this process is: hand automatable checks to machines and AI, and leave the parts requiring judgment, trade-offs, and accountability to people. Behind the data of doubled merged PRs and a rollback rate down more than 40% is the combined effect of process design and responsibility division, not just improved model capabilities.

One-line Summary

Anthropic shares the Claude Code team's code review process, using multi-agent collaboration and risk tiering to achieve doubled PRs with rollback rate down over 40%.

Paragraph Summary

Facing pressure as daily merged changes surged from 2,500 to 30,000, Anthropic's Claude Code team designed a code review process covering development-environment self-testing, CI checks, multi-agent AI review, security review, risk-tiered human approval, merge queues, and post-release monitoring. Core mechanisms include: using agents with fresh context for independent second opinions; multi-agent parallel review with voting to filter false positives; setting round limits, narrowing re-review scope, and requiring comments to be clear in one pass to avoid infinite loops; automatically determining risk levels from code changes to decide the number of human approvers; and merge queues re-running checks on the latest main branch to reduce combination risk. Ultimately, merged PRs doubled while the rollback rate dropped by more than 40%.