Search code examples
javasocketsconnectionapache-httpclient-4.xforceclose

How to force Apache HttpClient / HttpConnection to absolutely close tcp connection?


I use lot of http connection with Apache HttpClient (org.apache.httpcomponents 4.3.6) to test servers and I cannot force connections to close, even when I schedule HttpConnectionManager to httpClient.getConnectionManager().shutdown(); after 10 seconds in another thread.

httpClient.close() also doesn't help.

Connections can last for minutes or even hours.

I have tried custom SocketConfig and this also not helps:

 SocketConfig socketConfig = SocketConfig.custom()
                .setSoKeepAlive(false)
                .setSoLinger(5)
                .setSoReuseAddress(true)
                .setSoTimeout(5000)
                .setTcpNoDelay(true).build();

The way I am fetching the content:

  HttpUriRequest request = new HttpGet(url);
  HttpResponse response = httpClient.execute(request);

  try (InputStream in = response.getEntity().getContent()) {
       String result = IOUtils.toString(in, StandardCharsets.UTF_8);
       httpClient.close();
       return result;
  }

The way I am building HTTP Client:

    SocketConfig socketConfig = SocketConfig.custom()
            .setSoKeepAlive(false)
            .setSoLinger(configuration.getInt("proxy.list.socket.so.linger"))
            .setSoReuseAddress(true)
            .setSoTimeout(configuration.getInt("proxy.list.socket.so.timeout"))
            .setTcpNoDelay(true).build();

    HttpClientBuilder builder = HttpClientBuilder.create();
    builder.disableAutomaticRetries();
    builder.disableContentCompression();
    builder.disableCookieManagement();
    builder.disableRedirectHandling();
    builder.setConnectionReuseStrategy(new NoConnectionReuseStrategy());
    builder.setDefaultSocketConfig(socketConfig);

One of my prototypes of doing shutdown:

      shutdownExecutor.schedule(() -> {
            httpClient.getConnectionManager().closeExpiredConnections();
            httpClient.getConnectionManager().closeIdleConnections(1, TimeUnit.SECONDS);
            httpClient.getConnectionManager().shutdown();

            httpClient.notifyAll();
            try {
                httpClient.close();
            } catch (IOException ex) {

            }

        }, configuration.getInt("proxy.test.forced.timeout.seconds"), TimeUnit.SECONDS);

        String content = HttpContentFetcher.getAndCloseClient(url, httpClient);

Solution

  • RequestConfig has helped. Now it looks like all the connections are discarded in specified limits.

           RequestConfig config= RequestConfig.custom()
                    .setCircularRedirectsAllowed(false)
                    .setConnectionRequestTimeout(4000)
                    .setConnectTimeout(4000)
                    .setMaxRedirects(0)
                    .setRedirectsEnabled(false)
                    .setSocketTimeout(4000)
                    .setStaleConnectionCheckEnabled(true).build();
            request.setConfig(config);