ES Rest High Level Client throws SocketTimeoutException after being idle for sometime

Question

RestHighLevelClient is used to connect to ES 6.4(hosted on AWS) in a spring-boot app. When the app goes idle for some time, and request arrives, then the RestHighLevelClient throws SocketTimeoutException:

[Request processing failed; nested exception is org.springframework.data.elasticsearch.ElasticsearchException: Error while bulk for request: org.elasticsearch.action.bulk.BulkRequest@21511b6c] w
java.net.SocketTimeoutException: 5,000 milliseconds timeout on connection http-outgoing-38 [ACTIVE]
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.timeout(HttpAsyncRequestExecutor.java:387) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:92) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
at org.apache.http.impl.nio.client.InternalIODispatch.onTimeout(InternalIODispatch.java:39) ~[httpasyncclient-4.1.4.jar!/:4.1.4]
at org.apache.http.impl.nio.reactor.AbstractIODispatch.timeout(AbstractIODispatch.java:175) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionTimedOut(BaseIOReactor.java:263) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.AbstractIOReactor.timeoutCheck(AbstractIOReactor.java:492) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:213) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.11.jar!/:4.4.11]
at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

RestHighLevelClient is created using:

    @Bean
    RestHighLevelClient client() {

        ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(elasticsearchHostAndPort)
                .build();

        return RestClients.create(clientConfiguration).rest();
    }

spring-data-elasticsearch version 3.2.0.M2 is used.

Any hints/workarounds?

Answer 1

In RestClientBuilder.createHttpClient() the defaults for socket timeout and connection timeout are set to 30 and 10 seconds.

You can override this defaults by implementing RestClientBuilder.RequestConfigCallback and calling setRequestConfigCallback(...) on your RestHighLevelClient

We did something like

@Override
public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder builder) {
    return builder.setSocketTimeout(socketTimeout); // try to prevent SocketTimeoutException
}