OpenRouter Armor Review: Three Modes of Agent Tool-Call Interception and the Claude Code Loophole

Title

OpenRouter Armor Review: Three Modes of Agent Tool-Call Interception and the Claude Code Loophole

Meta Description

A deep dive into the OpenRouter Armor beta safety classifier: using the TypeSafe Jev model to vet agent tool calls, with hard_deny/soft_deny/allow three-tier rules. But in Claude Code compatibility mode it only flags without blocking—a self-built Messages client that ignores safeguard_results will execute flagged calls.

Article Content

When an agent deletes a folder you never mentioned on its own initiative, the problem is that there's no gate between the model and the shell to say "no." OpenRouter launched the beta version of Armor on October 15 to be that gate: a safety classifier sitting between the model and the application, checking every tool call against natural-language rules. Flagged calls are not executed, and the model is told which rule it hit.

But the documentation hides an exception, and it happens to surface in the scenario closest to Claude Code.

How Armor decides: a model, not a regex

For each tool call, OpenRouter sends the call content, your rules, and the conversation context to a decision model, which returns the probability that the call matches each rule, then aggregates that into a flagged or not_flagged verdict.

In its launch post, OpenRouter names the underlying model: Jev from TypeSafe. It's a structured decision model that returns typed answers with probabilities rather than free-form generated text. The current version is typesafe/jev-1.14.

Why a model instead of a list of rules? OpenRouter's claim is that "regex-based filtering is often insufficient to identify unwanted tool calls." The launch post also claims Armor outperforms Claude Code's built-in auto-mode classifier on both performance and cost. However, a review from explainx.ai notes that OpenRouter did not disclose the test set and numbers behind these conclusions. That's a fair question—performance claims about safety are just marketing speak without reproducible evaluation.

Structured decision models are a class of models trained specifically for classification, routing, or scoring tasks. They output typed structured results (such as enumerated labels with confidence probabilities) rather than free-form generated text. Compared with general-purpose LLMs, they are usually smaller, faster, and cheaper, with stable output formats that programs can parse directly. OpenRouter chose TypeSafe's Jev precisely for this: every tool call needs to be judged in milliseconds, and the results need comparable probability values to check against thresholds. This also defines the model's capability boundary—it excels at making judgments according to given rules but lacks general reasoning ability, and its performance drops noticeably when rules are written vaguely.

Three tiers of rules: hard_deny, soft_deny, and allow

The rule system has three tiers:

• hard_deny is for actions that "must never happen no matter what the user says";
• soft_deny is for actions that "usually shouldn't be done, but sometimes are genuinely reasonable requests." Once matched, they are blocked unless an allow rule also matches, or the user explicitly requests that operation on that target in the conversation;
• allow never blocks anything itself; it only defines exceptions to soft-deny.

Armor also has a built-in soft-deny targeting constraints the user has set in the conversation that haven't been lifted, such as "don't push until I've reviewed it." Each rule contains an id, an optional name, instructions, and an optional threshold. Deny rules default to a threshold of 0.6, and allow rules default to 0.7.

A practical detail is "mode": "audit": in this mode Armor scores every call but blocks nothing, which is suitable for tuning rules on real traffic first before switching to enforcement. Note that audit mode cannot be combined with claude_code_compatibility.

Where blocked calls go: three output_mode options

How interception results are presented depends on output_mode:

output_mode	Supported APIs	Form of a blocked call
text (default for Chat Completions and Responses)	Chat Completions, Responses	An <openrouter_armor_blocked> marker appears in the assistant text
custom_item	Responses	An openrouter:armor_blocked output item
claude_code_compatibility	Messages	The tool_use is preserved and marked as flagged in safeguard_results

In the first two modes, blocked calls never reach your code. The model is told that the call "was blocked by OpenRouter Armor under a certain rule and was not executed," and is asked not to retry with the same parameters.

The third mode behaves completely differently, and that's the core issue this article's title refers to.

Chat Completions, Responses, and Messages are the three main request formats in current LLM APIs. Chat Completions is OpenAI's earliest widely adopted chat completion interface and has the broadest ecosystem; Responses is OpenAI's newer interface, using a unified input/output item model to express text, tool calls, and multi-turn state; Messages is Anthropic's interface format, on which tools like Claude Code are built. The differences among the three directly determine where Armor can put interception results: Chat Completions only has a text channel, so it can only use the <openrouter_armor_blocked> marker; Responses has structured output items, so it can add an openrouter:armor_blocked type; the tool_use block in Messages is parsed and executed by the client itself, so Armor can only attach safeguard_results for the client's reference and cannot refuse execution on the client's behalf.

Claude Code mode: flag only, don't block

A Messages request carrying a safeguards entry of type dangerous_tool_use automatically selects this mode, with no plugins configuration required. The documentation is blunt: "Under claude_code_compatibility, it is your application that blocks the call." In the flow described by the docs, Claude Code is that application: it reads the flagged verdict, refuses to execute, and writes back a tool result with is_error.

The loophole is this: if you write your own Messages client and it never reads the verdict, the flagged tool_use will be executed directly. The documentation explicitly warns: "A Messages client that ignores safeguard_results will run the flagged tool_use." If you cannot confirm that the client will check, the official recommendation is to switch to "output_mode": "text".

If you do build your own Messages client, you must check before execution. safeguard_results is a top-level array, and each entry's status.tool_uses is keyed by tool-call id, containing type, outcome, and an optional explanation. A safe approach is: execute only when Armor has actually evaluated the call and the verdict is not_flagged; reject anything where the verdict is missing or is unavailable. To be clear, rejecting missing verdicts is the author's choice, not a mandatory documentation requirement, but it is consistent with the default direction of on_error in Armor plugin requests.

Closed or open on failure: the bounds of on_error

The decision model may time out, error out, or receive a request too large to judge. In that case the verdict is unavailable, and on_error determines the behavior: block (default) or log. But this setting does not apply to Claude Code mode—there, calls with an unavailable verdict are always allowed through with a visible verdict, leaving the decision to the client.

A few other boundary conditions worth noting:

• Armor does not yet support requests under HIPAA, Zero Data Retention, or data residency restrictions. Plugin requests in these scenarios return 403, while Claude Code requests are allowed through and marked unavailable, falling back to Claude Code's local auto-mode check. In other words, under zero data retention, Armor is not the one guarding the gate today.
• The decision model will produce false positives and false negatives. The per-rule threshold is the tuning knob.
• When loading the Claude Code compatibility rule set, Armor's median latency increases by 279 ms, and P90 increases by 334 ms.
• Evaluation is not billed during the beta, but OpenRouter says it will introduce pricing before general availability, and configuration fields, the safeguard_results object, and the openrouter:armor_blocked item type may all change.

The third point is a cost you pay on every tool call. In an agent turn with multiple consecutive tool calls, this latency adds up.

block and log in on_error correspond to the two failure strategies in distributed systems: fail-closed and fail-open. Fail-closed means denying by default when a dependency is unavailable—high safety but degraded availability; fail-open means allowing by default, ensuring business continuity but exposing risk to downstream. For agent safety checks, the choice depends on comparing the cost of false blocks against the cost of missed blocks: block will interrupt normal tasks when the decision model is flaky, while log may let dangerous calls through during the window when the checker is down. It's worth noting that both strategies only transfer risk rather than eliminate it. In practice, deployments usually need audit logs as well, so that behavior during fail-open periods can be traced after the fact.

Running Claude Code through Ori

Armor's Claude Code path goes through Ori, OpenRouter's launcher that runs the agent CLIs it supports on OpenRouter. It uses OAuth login instead of pasting API keys, and supports Claude Code, Codex, OpenCode, Cline, and others. The install script is curl -fsSL https://openrouter.ai/labs/ori/install.sh | bash—before piping it into a shell, it's worth reading the script first.

Then launch with a non-Anthropic model plus auto mode:

ori claude --model moonshotai/kimi-k2-0905 -- --permission-mode auto

Claude Code's auto mode uses a second model to review actions. According to GIGAZINE, through Ori, models on OpenRouter can use this approval mechanism, with Armor taking on the reviewer role.

Three things to do before you lock the door

If you want to try Armor, think through these three points first:

1. Start with "mode": "audit", read the verdicts on real traffic, and only allow blocking after you've confirmed the rules are reasonable.
2. On Chat Completions and Responses, deliberately choose on_error. block prevents calls from getting through when the checker fails; log avoids blocking the agent when the checker fails—it depends on your risk appetite.
3. If you own the Messages client, check safeguard_results before every tool_use, or simply switch to "output_mode": "text".

Armor is essentially a network hop inserted before every agent action, plus a decision about whether to be closed or open on failure. This brings up latency budgets, timeout handling, and system behavior when dependencies are unavailable—classic distributed systems design issues that now land on agent safety checks.

One question worth weighing: are you willing to pay about 300 milliseconds extra per agent tool call for a layer of checks guarded by a second model? Or do you trust permission prompts and your own eyes more?

One-line Summary

OpenRouter Armor uses a model to vet tool calls, but in Claude Code mode it only flags without blocking.

Paragraph Summary

OpenRouter has launched Armor in beta, inserting a safety classifier between the model and the application that checks every tool call against natural-language rules. It scores based on the TypeSafe Jev structured decision model, with rules in three tiers—hard_deny, soft_deny, and allow. Deny rules default to a threshold of 0.6, and an audit mode lets you observe before blocking. On Chat Completions and Responses, blocked calls never reach the code; but in Messages' claude_code_compatibility mode, Armor only marks calls as flagged, shifting blocking responsibility to the client—a self-built client that ignores safeguard_results will directly execute the flagged tool_use. In addition, the block/log settings of on_error don't apply to this mode, unavailable verdicts are always let through, and scenarios like zero data retention are outside Armor's coverage. After loading the Claude Code rule set, median latency increases by 279 ms, and you pay this cost on every tool call.