Search code examples
javasslconnection-poolingapache-httpclient-4.xapache-httpasyncclient

Apache AsyncHttpClient 4.1.4 creating new socket connection instead of reusing connection from Connection pool

We are using Apache AsyncHttpClient having below dependencies

[INFO] +- org.apache.httpcomponents:httpasyncclient:jar:4.1.4:compile
[INFO] |  +- org.apache.httpcomponents:httpcore:jar:4.4.10:compile
[INFO] |  +- org.apache.httpcomponents:httpcore-nio:jar:4.4.10:compile
[INFO] |  \- org.apache.httpcomponents:httpclient:jar:4.5.6:compile

We are noticing it is creating a new socket connection with downstream(having ssl and client auth enabled)for every request. Our expectation is to reuse the connection from pool.

Following are the debug logs 

2020:03:31:19:49:27.430 DEBUG I/O dispatcher 8 ManagedNHttpClientConnectionImpl  http-outgoing-55 127.0.0.1:51706<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.430 DEBUG I/O dispatcher 9 ManagedNHttpClientConnectionImpl  http-outgoing-56 127.0.0.1:51740<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.534 DEBUG I/O dispatcher 10 ManagedNHttpClientConnectionImpl  http-outgoing-57 127.0.0.1:51741<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.637 DEBUG I/O dispatcher 11 ManagedNHttpClientConnectionImpl  http-outgoing-58 127.0.0.1:51742<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.740 DEBUG I/O dispatcher 12 ManagedNHttpClientConnectionImpl  http-outgoing-59 127.0.0.1:51743<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.840 DEBUG I/O dispatcher 13 ManagedNHttpClientConnectionImpl  http-outgoing-60 127.0.0.1:51744<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:32.938 DEBUG I/O dispatcher 14 ManagedNHttpClientConnectionImpl  http-outgoing-61 127.0.0.1:51745<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.041 DEBUG I/O dispatcher 15 ManagedNHttpClientConnectionImpl  http-outgoing-62 127.0.0.1:51746<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.146 DEBUG I/O dispatcher 16 ManagedNHttpClientConnectionImpl  http-outgoing-63 127.0.0.1:51747<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.238 DEBUG I/O dispatcher 1 ManagedNHttpClientConnectionImpl  http-outgoing-64 127.0.0.1:51748<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.344 DEBUG I/O dispatcher 2 ManagedNHttpClientConnectionImpl  http-outgoing-65 127.0.0.1:51749<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.432 DEBUG I/O dispatcher 3 ManagedNHttpClientConnectionImpl  http-outgoing-66 127.0.0.1:51750<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.541 DEBUG I/O dispatcher 4 ManagedNHttpClientConnectionImpl  http-outgoing-67 127.0.0.1:51751<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.638 DEBUG I/O dispatcher 5 ManagedNHttpClientConnectionImpl  http-outgoing-68 127.0.0.1:51752<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500
2020:03:31:19:51:33.743 DEBUG I/O dispatcher 6 ManagedNHttpClientConnectionImpl  http-outgoing-69 127.0.0.1:51753<->127.0.0.1:8102[ACTIVE][rw:][ACTIVE][rw][NEED_UNWRAP][0][0][239]: Set timeout 500

Note: When used without SSL, it is working as per our expectation(reusing the connection from pool)

Following are our configs

      "sslEnabled": true,
      "host": "127.0.0.1",
      "port": 8102,
      "staleConnectionMonitorThreadName": "http-stale-connection-cleaner-thread",
      "publishMaxThreadPoolSize": 1,
      "defaultMaxThreadsPerRoute": 1,
      "maxThreadsPerRoute": 1,
      "connectionRequestTimeoutMs": 500,
      "connectionTimeoutMs": 500,
      "socketTimeoutMs": 500,
      "evictThreadSleepTimeMs" : 5000,
      "maxKeepAliveTimeMs" : 30000,
      "trustStorePath": "abc.jks",
      "trustStoreKey": "**",
      "keyStorePath": "xyx.jks",
      "keyStoreKey": "**"

Following are request headers

Content-Type: application/json
v-c-correlation-id: f7e046a9-e1f1-44f1-9fb4-dddcb973a951
v-c-username: xyz
Content-Length: 218
Host: 127.0.0.1:8102
Connection: Keep-Alive
User-Agent: Apache-HttpAsyncClient/4.1.4 (Java/1.8.0_201)

Following are response headers

"HTTP/1.1 200 OK[\r][\n]"
"Content-Type: application/json[\r][\n]"
"v-c-correlation-id: f7e046a9-e1f1-44f1-9fb4-dddcb973a951[\r][\n]"
"Content-Length: 0[\r][\n]"

Note: jdk version is 1.8

Solution

  • Apache HttpClient differentiate so called state-less and state-ful connections. Stateful connections are those that get created with a particular user identity or a security context. Two types are supported by HttpClient version 4.x and 5.x by default: connections with NTLM authentication and TLS connections with client authentication. Stateful connections while perfectly re-usable are considered sensitive. The connection pool manager will not lease those connections unless the user token in the HTTP execution context matches that the state of the connection kept alive in the pool.

    1. One execute multiple requests within the same execution context and thus make them share the same user identity. IMPORTANT: HttpContext instances MUST NOT be used concurrently! Some attributes stored in HttpContext are not threading safe! 

      CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
try {
    httpclient.start();
    HttpClientContext clientContext = HttpClientContext.create();
    HttpGet request1 = new HttpGet("http://httpbin.org/get");
    Future<HttpResponse> future1 = httpclient.execute(request1, clientContext, null);
    HttpResponse response1 = future1.get();
    System.out.println("Response: " + response1.getStatusLine());
    HttpGet request2 = new HttpGet("http://httpbin.org/get");
    Future<HttpResponse> future2 = httpclient.execute(request2, clientContext, null);
    HttpResponse response2 = future2.get();
    System.out.println("Response: " + response2.getStatusLine());
} finally {
    System.out.println("Shutting down");
    httpclient.close();
}

    2. Use different HttpContext instances for related requests but manually assign the same user token to those requests

      CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
try {
    httpclient.start();
    HttpClientContext clientContext1 = HttpClientContext.create();
    clientContext1.setUserToken("appuser");
    HttpGet request1 = new HttpGet("http://httpbin.org/get");
    Future<HttpResponse> future1 = httpclient.execute(request1, clientContext1, null);
    HttpResponse response1 = future1.get();
    System.out.println("Response: " + response1.getStatusLine());
    HttpClientContext clientContext2 = HttpClientContext.create();
    clientContext2.setUserToken("appuser");
    HttpGet request2 = new HttpGet("http://httpbin.org/get");
    Future<HttpResponse> future2 = httpclient.execute(request2, clientContext2, null);
    HttpResponse response2 = future2.get();
    System.out.println("Response: " + response2.getStatusLine());
} finally {
    System.out.println("Shutting down");
    httpclient.close();
}

    3. If the client endpoint does not support different user identities (there is a common security context shared by all HTTP requests) one can simply disable connection state management

      CloseableHttpAsyncClient httpclient = HttpAsyncClients.custom()
        .disableConnectionState()
        .build();