Circuit Breaker

The circuit-breaker pattern is a fault-tolerance pattern borrowed from distributed systems engineering that automatically cuts off call chains when a dependent service fails consecutively beyond a threshold, to prevent cascading failures and resource exhaustion.

The circuit breaker pattern originates from the fault-tolerance design philosophy of microservice architecture, automatically interrupting and rolling back when the failure rate exceeds a threshold.

The circuit breaker pattern originates from fault-tolerance design in distributed systems, was systematically described by Michael Nygard in "Release It!", and cuts off calls to downstream services by monitoring the number of consecutive failures.

Configuring a backup model, exponential backoff retries, subscribing to the official status page, and circuit breaking with fallback are mitigation measures for the risk of depending on a single model.

Loop engineering borrows the circuit breaker mechanism, using the "maximum retry count" as the threshold: once the generator AI fails to pass the evaluator's tests within the set number of rounds, the system archives the current state and marks it as requiring human intervention.

The loop engineering system pairs a maximum retry count as a circuit breaker: if it can't be fixed, archive it and call a human, preventing the loop from spinning idly.

Circuit breakers are a classic distributed pattern but are rarely implemented in early agent frameworks.

Circuit breakers in microservices apply to agents: when an agent or downstream service fails repeatedly, the circuit opens, preventing subsequent retries from continuing to waste resources and exacerbate congestion.

The author implemented a circuit breaker with a single Postgres table, recording workflow_id, step_id, fail_count, last_fail, cleared_by, and cleared_at; two consecutive failures stop that step and send an ntfy alert, until it is reset after human confirmation. Since this mechanism was built, it has been triggered four times, catching issues such as rate limiting, malformed URLs, portal processing delays, and network timeouts.

The production-readiness checklist includes four hard requirements: external calls must have a circuit breaker (retry at most 2 times, then stop and alert); structured data must be passed between agents; irreversible operations must be followed by a verification step; every call must have a complete, queryable execution log.

In LLM repair loops, the recommended typical circuit breaker parameters are: 5 consecutive failures trigger the break, a 30-60 second cooldown, and rate-limited probing during the half-open phase.

When using a circuit breaker on an LLM pipeline, if the model's validation failure rate spikes within a short period, it indicates a systemic problem, and continuing to call it will only keep producing waste and burning token costs.

The circuit breaker originates from the field of distributed systems; its logic is to temporarily cut off calls when a dependency's consecutive failures reach a threshold, and a typical state machine includes three states: closed, open, and half-open.

Circuit breakers and retries are complementary: retries handle occasional failures, and circuit breakers handle systemic failures.

The author advocates managing LLMs as noisy machines, following a four-step process: draw the blueprint (define the data structure contract), inspect each piece (schema validation), rework with defect descriptions (repair loop), and shut it down when the machine breaks (circuit breaker).

The structure of the circuit breaker consists of three parts: an append-only ledger, a gate before the call, and a settlement after the call.

Failed model call attempts usually still consume input tokens, so even errored requests must be recorded, otherwise the ledgered spend will be lower than the actual bill and the circuit breaker will close only at the wrong point in time.

The emergency braking mechanism is similar to circuit breakers and interruptible execution patterns in distributed systems; its implementation requires the agent runtime to periodically check an abort flag and to support hooks before and after tool calls to block high-risk actions.

A maximum number of round trips between agents, a maximum total duration, and a maximum cost should be set for each workflow, with mandatory escalation to a human when any limit is exceeded.

Recommended agent guardrail patterns include: a default-deny outbound whitelist, blocking offensive payloads, and triggering a circuit breaker to terminate the session by throwing an exception when an agent is repeatedly rejected on the same host.