Circuit Breaker

A circuit breaker is a component that wraps calls to a remote dependency and stops making those calls when the dependency is failing. While the breaker is open, calls fail immediately without touching the network. After a wait, the breaker lets a few trial calls through to test whether the dependency has recovered.

The problem it solves

When a dependency is down or very slow, every caller keeps sending requests and waiting for them to time out. This has three effects:

• Callers hold threads, connections, or memory for the full timeout duration, so their own capacity is consumed.
• The failing dependency receives continuous load and cannot recover.
• Users wait for the timeout before seeing an error.

Timeouts and retries do not solve this: retries add load, and timeouts still cost a full wait per request. A circuit breaker detects sustained failure and switches to failing fast: returning an error, or a fallback, in microseconds.

States

A circuit breaker is a state machine with three states.

stateDiagram-v2
    [*] --> Closed
    Closed --> Open: failure threshold exceeded
    Open --> HalfOpen: open timeout elapsed
    HalfOpen --> Closed: trial calls succeed
    HalfOpen --> Open: a trial call fails

The half-open state prevents two problems: a breaker that never closes again, and a full flood of traffic hitting a dependency that has only just started to recover.

What triggers opening

The breaker decides using recent call outcomes. Common criteria:

• Consecutive failures: open after N failures in a row. Simple, but sensitive to a single success resetting the count.
• Failure rate over a window: open when the failure percentage exceeds a threshold (for example 50%) within a sliding window, provided a minimum number of calls occurred in that window. The minimum avoids opening after 1 failure out of 2 calls.
• Slow-call rate: count calls slower than a threshold as failures, since a slow dependency is as harmful as a failing one.

Windows can be count-based (last N calls) or time-based (last N seconds).

Example: a count-based window of the last 20 calls, a minimum of 10 calls, and a failure-rate threshold of 50%. After 4 calls with 3 failures the rate is 75%, but fewer than 10 calls were recorded, so the breaker stays closed. After 12 calls with 6 failures the rate is 50% and the breaker opens. A successful call in a 20-call window only lowers the rate gradually, unlike a consecutive-failure counter that a single success resets to zero.

What counts as a failure needs care:

• Count: timeouts, connection errors, 5xx responses, UNAVAILABLE.
• Do not count: 4xx client errors such as validation failures. They indicate a bad request, not an unhealthy dependency.

Configuration parameters

Open duration is often fixed, for example tens of seconds. It can also grow with repeated re-opens, similar to backoff.

Per-dependency breakers

Use one breaker per dependency, and often per operation or endpoint, not one global breaker. A single breaker shared across dependencies opens for all of them when one fails. Breakers are usually kept per caller instance, so each instance decides from its own observations. This needs no coordination and no shared state, but instances learn about the failure separately.

Placement options:

• In-process library: Resilience4j (Java), Polly (.NET), and similar libraries wrap calls in the client. In Java, a breaker can be a small class like the one below. Hystrix was an early widely used implementation and is in maintenance mode.
• Service mesh or sidecar proxy: proxies such as Envoy provide outlier detection, which ejects unhealthy hosts from a load-balancing pool, and connection and request limits that act as circuit breaking. See service mesh and sidecars.

Fallbacks

When the breaker rejects a call, the caller decides what to do:

• Return a cached or stale value.
• Return a default or degraded response, such as an empty recommendation list.
• Queue the request for later processing.
• Return a clear error to the client.

The choice depends on the operation. Reads often have a sensible fallback. Payment authorization usually does not. See bulkhead, fallback, and load shedding and health checks and graceful degradation.

Java implementation

The following breaker is count-based with consecutive failures. It takes the current time as an argument (a plain long tick, for example milliseconds), so the example is deterministic and testable. The state is a sealed interface with one record per state, and a call returns a sealed result: success, rejected by the open breaker, or failed in the dependency.

import java.util.function.Supplier;

public class Main {
    sealed interface State {
        record Closed() implements State {}
        record Open(long since) implements State {}
        record HalfOpen() implements State {}
    }

    sealed interface Result<T> {
        record Success<T>(T value) implements Result<T> {}
        record Rejected<T>() implements Result<T> {}
        record Failed<T>() implements Result<T> {}
    }

    /** Thrown by an operation to report that the dependency failed. */
    static class DependencyException extends RuntimeException {}

    static class CircuitBreaker {
        private State state = new State.Closed();
        private int consecutiveFailures;
        private final int failureThreshold;
        private final long openDuration;

        CircuitBreaker(int failureThreshold, long openDuration) {
            if (failureThreshold < 1 || openDuration < 0) {
                throw new IllegalArgumentException("invalid configuration");
            }
            this.failureThreshold = failureThreshold;
            this.openDuration = openDuration;
        }

        State state() {
            return state;
        }

        /** Runs op if the breaker allows it. now is the current time. */
        <T> Result<T> call(long now, Supplier<T> op) {
            if (state instanceof State.Open open) {
                if (now - open.since() >= openDuration) {
                    state = new State.HalfOpen(); // allow a trial call
                } else {
                    return new Result.Rejected<>();
                }
            }
            try {
                T value = op.get();
                consecutiveFailures = 0;
                state = new State.Closed();
                return new Result.Success<>(value);
            } catch (DependencyException e) {
                consecutiveFailures++;
                if (state instanceof State.HalfOpen || consecutiveFailures >= failureThreshold) {
                    state = new State.Open(now);
                }
                return new Result.Failed<>();
            }
        }
    }

    public static void main(String[] args) {
        var breaker = new CircuitBreaker(3, 100);

        // (time, dependency healthy at that time)
        long[] times = {0, 10, 20, 30, 130, 240, 250};
        boolean[] healthy = {false, false, false, false, false, true, true};

        for (int i = 0; i < times.length; i++) {
            boolean ok = healthy[i];
            Result<String> result = breaker.call(times[i], () -> {
                if (!ok) {
                    throw new DependencyException();
                }
                return "ok";
            });
            System.out.printf("t=%3d state=%s result=%s%n", times[i], breaker.state(), result);
        }
    }
}

t=  0 state=Closed[] result=Failed[]
t= 10 state=Closed[] result=Failed[]
t= 20 state=Open[since=20] result=Failed[]
t= 30 state=Open[since=20] result=Rejected[]
t=130 state=Open[since=130] result=Failed[]
t=240 state=Closed[] result=Success[value=ok]
t=250 state=Closed[] result=Success[value=ok]

Trace: three consecutive failures open the breaker at t=20. The call at t=30 is rejected without invoking the operation. At t=130 the open duration has elapsed, so one trial call is made in the half-open state. It fails, so the breaker re-opens. At t=240 the trial call succeeds and the breaker closes.

This class is not thread-safe. In a concurrent service the state would be held behind a lock or in an AtomicReference, and half-open would admit only a limited number of concurrent trial calls (for example with a Semaphore).

Trade-offs and pitfalls

• Thresholds are workload dependent. A low threshold trips on normal noise. A high one reacts too late. Rate-based windows with a minimum call count tolerate noise better than raw failure counts.
• Half-open concurrency. If every waiting request is allowed through when the open duration elapses, the dependency receives a burst exactly when it is weakest. The number of concurrent trial calls must be limited, and rejected callers keep failing fast while trials are in flight.
• Per-instance state. Each caller instance opens and closes on its own schedule. With many instances, trial calls from all of them can add up to substantial load. A random component in the open duration spreads them out.
• Low-traffic dependencies produce too few samples for a meaningful failure rate. A breaker on a rarely called endpoint may stay closed or flap.
• Retries and breakers interact. Place the breaker inside the retry loop, so each attempt passes through it. Once open, remaining retries fail immediately instead of adding load.
• Rejected calls need handling. An open breaker moves failure handling to the caller. Without a fallback or a clear error, the effect is the same failure, only faster.
• Not a fix for capacity problems. A breaker reacts to failure and does not limit concurrency for a healthy but slow dependency. Combine with bulkheads and timeouts.
• Monitoring. State transitions should be logged and exported as metrics, because an open breaker signals a real incident.

Interview questions

• What are the three states of a circuit breaker? Closed (calls pass and outcomes are recorded), open (calls are rejected immediately), and half-open (a limited number of trial calls test whether the dependency has recovered).
• Why is a half-open state needed? It allows recovery to be detected gradually. Without it, the breaker would either stay open forever or return to full traffic against a dependency that may still be failing.
• Should a 404 or 400 response count as a breaker failure? No. Client errors reflect the request, not the health of the dependency. Timeouts, connection failures, and 5xx responses count.
• How does a circuit breaker differ from a retry? A retry repeats a single failed call. A breaker aggregates results across calls and stops making calls altogether when failures are sustained, which protects both the caller and the dependency.
• Where should a breaker be placed relative to retries? Inside the retry loop, so that each attempt is checked by the breaker and retries stop generating load once it opens.