Hi I am using Apache camel http4 component to send a https request with keepAlive=true but when i see the netstat after sending multiple requests I see each request opens a new TCP port to the peer.
I feel this should not be the usual behavior of keepAlive transactions, why the same TCP port is not being reused for communicating with the server and how can that be achieved if at all it can be.
Turns out this is not a keep alive problem; connections are in fact kept alive properly. The problem is that the connections as part of the pool managed by default PoolingHttpClientConnectionManager were not being reused. Both things could've been easily seen by enabling the logging for Apache's HttpClient (that's used under the hood):
Keep-Alive is being used:
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "HTTP/1.1 200 OK[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Keep-Alive: timeout=5, max=300[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Server: Apache-Coyote/1.1[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Content-Encoding: gzip[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Vary: Accept-Encoding[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Cluster-Id: A[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Date: Wed, 19 Dec 2018 06:59:17 GMT[\r][\n]" 2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Content-Type: text/xml[\r][\n]" 2018/12/19 07:59:17:471 CET [DEBUG] wire - http-outgoing-7 << "Content-Length: 239[\r][\n]" 2018/12/19 07:59:17:471 CET [DEBUG] wire - http-outgoing-7 << "[\r][\n]" Connections not being reused:
2018/12/19 08:00:08:240 CET [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://someurl.com:443][total kept alive: 1; route allocated: 1 of 1; total allocated: 1 of 1] 2018/12/19 08:00:08:240 CET [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-7: Close connection 2018/12/19 08:00:08:242 CET [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 8][route: {s}->https://someurl.com:443][total kept alive: 0; route allocated: 1 of 1; total allocated: 1 of 1] Note that one can easily enable logging for HttpClient by passing some arguments to JVM at startup.
So, why are connections not being reused then? This is due to the fact that SSL is being used and the PoolingHttpClientConnectionManager used by Apache's HttpClient not allowing reuse of a connection in case the user principal from the existing connection differs from the requested connection (facilitated by DefaultUserTokenHandler). See also e.g. this Stackoverflow post. Solution is to implement a custom UserTokenHandler (or use the NullTokenHandler if that's sufficient) and configure the HttpClientBuilder accordingly.