Resilience4j circuit breaker does not switch back to closed state from half open after the downstream system is up again

Question

I have two micro-services, order-service, and inventory-service. The order-service makes a call to inventory-service to check if ordered items are in stock. An order is placed only if all the items in the order request are in stock. If the inventory-service is down or slow, the circuit breaker part triggers. The case of an order not being placed because of a missing item is not a failure case for the circuit breaker. This works well as intended only until the circuit has never switched from closed state to half open state.

What I mean by that is if more than 5 consecutive orders cannot be placed because of a missing item, the circuit does not switch to open state. This is as expected. If the inventory-service is brought down, 3 more failed requests cause the circuit to move to open and subsequently to half open state. This also is as expected. However when the inventory-service comes up again, and the only requests that are made are requests containing one or more items not in stock, the circuit remains in half_open state continuously. This is not ok. A missing item in an order is a success case and should increment the successful buffered call count, but it doesn't. Looking at the actuator info, it looks like these calls are not counted either as failure or as success cases. What am I doing wrong.

Note -- if I make sufficient number of calls where order gets placed, then the circuit switches to closed again. That's ok but shouldn't the case of ignored exception count as a success case even if the only calls that are made are those with one or missing items.

Following are the properties of my circuit breaker in the calling microservice.

resilience4j.circuitbreaker.instances.inventory.register-health-indicator=true
resilience4j.circuitbreaker.instances.inventory.event-consumer-buffer-size=10
resilience4j.circuitbreaker.instances.inventory.sliding-window-type=COUNT_BASED
resilience4j.circuitbreaker.instances.inventory.sliding-window-size=5
resilience4j.circuitbreaker.instances.inventory.failure-rate-threshold=50
resilience4j.circuitbreaker.instances.inventory.wait-duration-in-open-state=5s
resilience4j.circuitbreaker.instances.inventory.permitted-number-of-calls-in-half-open-state=3
resilience4j.circuitbreaker.instances.inventory.automatic-transition-from-open-to-half-open-enabled=true
resilience4j.circuitbreaker.instances.inventory.ignore-exceptions=com.mayrevision.orderservice.exception.OrderItemNotFoundException

I have a custom exception handler for OrderItemNotFoundException.

@ControllerAdvice
@ResponseStatus
public class OrderItemNotFoundExceptionHandler extends ResponseEntityExceptionHandler {
    @ExceptionHandler(OrderItemNotFoundException.class)
    public ResponseEntity<ErrorResponse> getErrorMessage(OrderItemNotFoundException exception) {
        ErrorResponse response = new ErrorResponse(HttpStatus.NOT_ACCEPTABLE, exception.getMessage());
        return ResponseEntity.status(HttpStatus.NOT_ACCEPTABLE).body(response);
    }
}

The controller code is --

@PostMapping
    @ResponseStatus(HttpStatus.CREATED)
    @CircuitBreaker(name = "inventory", fallbackMethod = "fallBackMethod")
    public String placeOrder(@RequestBody OrderRequest orderRequest) throws OrderItemNotFoundException {
        orderService.placeOrder(orderRequest);
        return "Order placed successfully";
    }

    public String fallBackMethod(OrderRequest orderRequest, RuntimeException runtimeException) {
        return "The order could not be placed. Please try back after some time.";
    }

Edit -- Edited resilience4j.circuitbreaker.instances.inventory.ignore-exceptions[0]=com.mayrevision.orderservice.exception.OrderItemNotFoundException to resilience4j.circuitbreaker.instances.inventory.ignore-exceptions=com.mayrevision.orderservice.exception.OrderItemNotFoundException in application.properties.

Answer 1

Looks like this is how the exception mechanism in Resilience4j is designed to work. If I want the exception to be treated as a success case in all the cases (including the case mentioned above), I should catch it. So I changed the code as follows --

    @PostMapping
    @CircuitBreaker(name = "inventory", fallbackMethod = "fallBackMethod")
    public CompletableFuture<ResponseEntity<StatusResponse>> placeOrder(@RequestBody OrderRequest orderRequest) {
        return  CompletableFuture.supplyAsync(() -> {
            try {
                String message = orderService.placeOrder(orderRequest);
                StatusResponse response = new StatusResponse(HttpStatus.CREATED, message);
                return ResponseEntity.status(HttpStatus.CREATED).body(response);
            } catch (OrderItemNotFoundException e) {
                return ResponseEntity.status(HttpStatus.NOT_ACCEPTABLE)
                        .body(new StatusResponse(HttpStatus.NOT_ACCEPTABLE, e.getMessage()));
            }
        });
    }

 public CompletableFuture<ResponseEntity<StatusResponse>> fallBackMethod(OrderRequest orderRequest, RuntimeException e) {
        return CompletableFuture.supplyAsync(() -> {
            //some logic
        });
    }

This is working close to expected.