Search code examples
javaspring-bootspring-cloudhystrixspring-cloud-feign

Spring Cloud Feign client with Hystrix circuit-breaker timeout defaults in 2 seconds

Reproducible through a project on GitHub: spring-cloud-feign-hystrix-timeout-problem

I am using Spring Boot 2.3.1.RELEASE with Spring Cloud Hoxton.SR6. Namely Feign client and Hystrix without Zuul and Eureka to fetch REST responses.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-dependencies</artifactId>
   <version>Hoxton.SR6</version>
   <type>pom</type>
   <scope>import</scope>
</dependency>

Then I use use the following dependencies on top of Spring Boot 2.3.1.RELEASE and Spring Cloud Hoxton.SR6:

  • org.springframework.boot:spring-boot-starter
  • org.springframework.boot:spring-boot-starter-web
  • org.springframework.cloud:spring-cloud-starter-openfeign
  • org.springframework.cloud:spring-cloud-starter-netflix-hystrix

I enable @EnableFeignClients and @EnableCircuitBreaker and use a @FeignClient with a simple fallback to log and rethrow an exception:

@FeignClient(name="my-feign", url = "${feign.url}", fallbackFactory = MyFallbackFactory.class) {
public interface MyFeignClient {

    @PostMapping(value = "/api/dto")
    postDto(@RequestBody Dto dto);
}

With the following application.yml the timeout is around 1 second because Hystrix defaults to the very same value:

feign:
  hystrix:
    enabled: true
  client:
    config:
      default:
        connectTimeout: 5000
        readTimeout: 5000

11:52:05.493 INFO 10200 --- [nio-8060-exec-2] com.mycompany.rest.MyController : Calling REST right now!

11:52:06.538 ERROR 24044 --- [nio-8060-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.netflix.hystrix.exception.HystrixRuntimeException: MyFeignClient#postDto(Dto) timed-out and fallback failed.] with root cause

What I tried?

As long as I add the following lines to increase the timeouts to 60 seconds, the timeout become effectively around 2 seconds:

hystrix:
  command:
    default:
      execution:
        timeout:
          enabled: true
        isolation:
          thread:
            timeoutInMilliseconds: 60000

11:53:33.590 INFO 16052 --- [nio-8060-exec-2] com.mycompany.rest.MyController : Calling REST right now!

11:53:35.614 ERROR 16052 --- [nio-8060-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.netflix.hystrix.exception.HystrixRuntimeException: MyFeignClient#postDto(Dto) failed and fallback failed.] with root cause

The call falls into fallback right in 2 seconds as long as the Hystrix read/connect timeouts have been increased. However I expect to achieve 5 seconds as long as I declared it in feign.client.config.default... timeouts. I feel I am missing another configuration.

Q: How to increase the timeout?

Edit:

Solution

  • Your configuration is correct and what you are describing is expected behavior. The issue here is that exception with Connection refused isn't thrown after your configured timeout -- 10 seconds. Instead, it is thrown immediately after Java's internal socket implementation finds that server host is not reachable. In simplest case, the server you are calling isn't up and running.

    As to why there is a second increase after you set the hystrix timeout, you can debug hystrix's call stack to find out that HystrixRuntimeException isn't being thrown in the same order.

    Before your custom hystrix timeout, hystrix had a default timeout of 1 second, which means this runtime exception is always thrown once one second since execution has lapsed, regardless whether the request succeeds or fails. So in your case, Connection refused could very likely happen after HystrixTimeoutException. After you set the timeout be to longer than that of feign client, HystrixTimeoutException only gets created after a feign exception is thrown (due to "Connection refused"), hence the delay.

    // 'cause' should be different due to timing
public HystrixRuntimeException(... Exception cause, Throwable fallbackException)

    To simulate the timeout, I'd say you could force a timeout on the server, such as Thread.sleep(6000) to halt the execution on server side, or simply do a breakpoint while on debugger.