Circuit Breaker

For introduction, see How to Fail Fast on Recurring Failures.

Description

The circuit breaker is a simple state machine:

The circuit breaker starts closed. In this state, the circuit breaker maintains a rolling window of recent invocations. For each invocation, the rolling window tracks whether it finished successfully or failed.

The rolling window must be full to take any state transition decision. For example, if the rolling window has size 10, a closed circuit breaker always allows at least 10 invocations.

If the rolling window contains a number of failures higher than the configured ratio, a closed circuit breaker moves to open. When the circuit breaker is open, invocations are not allowed to proceed. Instead, the circuit breaker fails fast and throws CircuitBreakerOpenException.

For example, if the rolling window has size 10 and the failure rate is 0.5, it means that 5 invocations out of the most recent 10 invocations must fail for the circuit breaker to move to open.

After some time, an open circuit breaker moves to half-open to determine whether failing fast is still appropriate. A half-open circuit breaker allows some configured number of probe attempts to proceed. If all of them succeed, the circuit breaker moves to closed and invocations are allowed again. If some probe invocations fail, the circuit breaker moves back to open and invocations are prevented.

Successes and failures are determined from the method result. More specifically:

If the guarded method returns normally, it is treated as success.
Otherwise, if the guarded method throws an exception whose type is assignable to any of the skipOn types (see skipOn), it is treated as success.
Otherwise, if the guarded method throws an exception whose type is assignable to any of the failOn types (see failOn), it is treated as failure.
Otherwise, it is treated as success.

Lifecycle

Circuit breaker needs to maintain some state between invocations: the number of recent successful invocations, the number of recent failed invocations, and so on. This state is a singleton, irrespective of the lifecycle of the bean that uses the @CircuitBreaker annotation.

More specifically, the circuit breaker state is uniquely identified by the combination of the bean class (java.lang.Class) and the method object (java.lang.reflect.Method) representing the guarded method.

For example, if there’s a guarded method doWork on a bean which is @RequestScoped, each request will have its own instance of the bean, but all invocations of doWork will share the same circuit breaker state.

Interactions with Other Strategies

See How to Use Multiple Strategies for an overview of how fault tolerance strategies are nested.

If @Fallback is used with @CircuitBreaker, the fallback method or handler may be invoked if a CircuitBreakerOpenException is thrown, depending on the fallback configuration.

If @Retry is used with @RateLimit, each retry attempt is processed by the circuit breaker as an independent invocation. If CircuitBreakerOpenException is thrown, the execution may be retried, depending on how retry is configured.

Configuration

There are 7 configuration options, corresponding to the 7 members of the @CircuitBreaker annotation.

`requestVolumeThreshold`

Type: int

Default: 20

The size of the rolling window. That is, the number of recent consecutive invocations tracked by a closed circuit breaker.

`failureRatio`

Type: double

Default: 0.5

The ratio of failures in the rolling window that causes a closed circuit breaker to move to open.

`delay` + `delayUnit`

Type: long + ChronoUnit

Default: 5000 millis, or 5 seconds

The delay after which an open circuit breaker moves to half-open.

`successThreshold`

Type: int

Default: 1

The number of probe invocations allowed when the circuit breaker is half-open. If they all succeed, the circuit breaker moves to closed, otherwise it moves back to open.

`failOn`

Type: Class<? extends Throwable>[]

Default: {Throwable.class}

Set of exception types that the circuit breaker considers failures.

`skipOn`

Type: Class<? extends Throwable>[]

Default: {}

Set of exception types that the circuit breaker considers success. This configuration takes precedence over failOn.

Metrics

Circuit breaker exposes the following metrics:

Name

ft.circuitbreaker.calls.total

Type

Counter

Unit

None

Description

The number of times the circuit breaker logic was run. This is usually once per method call, but may be more than once if the method call is retried.

Tags

method - the fully qualified method name
circuitBreakerResult = [success|failure|circuitBreakerOpen] - the result of the method call, as considered by the circuit breaker
- success - the method ran and was successful
- failure - the method ran and failed
- circuitBreakerOpen - the method did not run because the circuit breaker was in open or half-open state

Name

ft.circuitbreaker.state.total

Type

Gauge<Long>

Unit

Nanoseconds

Description

Amount of time the circuit breaker has spent in each state

Tags

method - the fully qualified method name
state = [open|closed|halfOpen] - the circuit breaker state

Notes

Although this metric is a Gauge, its value increases monotonically.

Name

ft.circuitbreaker.opened.total

Type

Counter

Unit

None

Description

Number of times the circuit breaker has moved from closed state to open state

Tags

method - the fully qualified method name

Name

ft.circuitbreaker.state.current

This is an additional feature of SmallRye Fault Tolerance and is not specified by MicroProfile Fault Tolerance.

Type

Gauge<Long> (0 or 1)

Unit

None

Description

Whether the circuit breaker is currently in given state (1) or not (0)

Tags

method - the fully qualified method name
state = [open|closed|halfOpen] - the circuit breaker state

See the Metrics reference guide for general metrics information.

Extra Features

Circuit Breaker Maintenance

This is an additional feature of SmallRye Fault Tolerance and is not specified by MicroProfile Fault Tolerance.

It is sometimes useful to see the circuit breaker status from within the application, or reset it to the initial state. This is possible in two steps:

Give the circuit breaker a name by annotating the guarded method with @CircuitBreakerName:
```
@ApplicationScoped
public class MyService {
    @CircuitBreaker
    @CircuitBreakerName("hello-cb") (1)
    public String hello() {
        ...
    }
}
```
1 The circuit breaker guarding the MyService.hello method is given a name hello-cb.

Inject CircuitBreakerMaintenance and call its methods:

@ApplicationScoped
public class Example {
    @Inject
    CircuitBreakerMaintenance maintenance;

    public void test() {
        System.out.println("Circuit breaker state: "
            + maintenance.currentState("hello-cb")); (1)
        maintenance.resetAll(); (2)
    }
}

1	Obtains current circuit breaker state.
2	Resets all circuit breakers to the initial state.

The CircuitBreakerMaintenance interface provides 4 methods:

currentState(name): returns current state of given circuit breaker. The return type CircuitBreakerState is an enum with 3 values: CLOSED, OPEN, HALF_OPEN.
onStateChange(name, callback): registers a callback that will be called when given circuit breaker changes state.
reset(name): resets given circuit breaker to the initial state.
resetAll(): resets all circuit breakers in the application to the initial state.

See the javadoc of those methods for more information.

Inspecting Exception Cause Chains

This is an additional feature of SmallRye Fault Tolerance and is not specified by MicroProfile Fault Tolerance.

This feature only works in the non-compatible mode.

The @CircuitBreaker annotation can specify that certain exceptions should be treated as failures (failOn) and others as successes (skipOn). The specification limits this to inspecting the actual exception that was thrown. However, in many cases, exceptions are wrapped and the exception the user wants to decide on is only present in the cause chain.

For that reason, in the non-compatible mode, if the actual thrown exception isn’t known failure or known success, SmallRye Fault Tolerance inspects the cause chain. To be specific, in case a @CircuitBreaker method throws an exception, the decision process is:

If the skipOn exceptions are not default and the exception is assignable to one of the skipOn exceptions, the circuit breaker treats it as a success.
Otherwise, if the failOn exceptions are not default and the exception is assignable to one of the failOn exceptions, the circuit breaker treats it as a failure.
Otherwise, if the exception is assignable to one of the skipOn exceptions or its cause chain contains an exception assignable to one of the skipOn exceptions, the circuit breaker treats it as a success.
Otherwise, if the exception is assignable to one of the failOn exceptions or its cause chain contains an exception assignable to one of the failOn exceptions, the circuit breaker treats it as a failure.
Otherwise, the exception is treated as a success.

For example:

@CircuitBreaker(requestVolumeThreshold = 10,
    skipOn = ExpectedOutcomeException.class, (1)
    failOn = IOException.class) (2)
public Result doSomething() {
    ...
}

1	If `doSomething` throws an `ExpectedOutcomeException`, or a `WrapperException` whose cause is `ExpectedOutcomeException`, the circuit breaker treats it as a success.
2	If `doSomething` throws an `IOException`, or a `WrapperException` whose cause is `IOException`, the circuit breaker treats it as a failure.

@CircuitBreaker(requestVolumeThreshold = 10,
    skipOn = ExpectedOutcomeException.class) (1) (2)
public Result doSomething() {
    ...
}

1	If `doSomething` throws an `ExpectedOutcomeException`, or a `WrapperException` whose cause is `ExpectedOutcomeException`, the circuit breaker treats it as a success.
2	There’s no `failOn`, so the 2nd step in the algorithm above is skipped. This is what turns the `WrapperException` whose cause is `ExpectedOutcomeException` into a success.

Circuit Breaker

Description

Lifecycle

Interactions with Other Strategies

Configuration

requestVolumeThreshold

failureRatio

delay + delayUnit

successThreshold

failOn

skipOn

Metrics

Extra Features

Circuit Breaker Maintenance

Inspecting Exception Cause Chains

`requestVolumeThreshold`

`failureRatio`

`delay` + `delayUnit`

`successThreshold`

`failOn`

`skipOn`