Java commons-httpclient: Testing timeout values

Question

Abstract: how do devs Integration TEST timeouts for http requests?

Backstory: My team is having issues related to unusually long lasting HTTP web requests. We use the commons-httpclient version 3 by Apache. The code looks similar to this:

PostMethod post = new PostMethod(endpoint);
post.getParams().setSoTimeout(someInt);
httpClient.executeMethod(post);

The time to complete this request is usually acceptable (2 seconds or so), but occasionally, we will see 50-60 second requests despite having our SO timeout set to 4 seconds. This prompted me to do some research and found that most people are setting Connection Timeouts ANNNNND SO timeouts. It appears that SO timeouts should be set lower (as they simply time the distance between bytes in transit) and the the connection timeout is what we originally planned to use (i.e. initial delay between request and 1st byte returned). Here is the code we scraped and plan on using:

httpClient.getHttpConnectionManager().getParams()
                  .setConnectionTimeout(someInt);

httpClient.getHttpConnectionManager().getParams()
                  .setSoTimeout(someInt);

The main pain here is that we are unable to integration test this change. More precisely, we are confused on how to integration test the delays coming from a socket connection to a foreign server. After digging through the commons-httpclient, I see protected and private classes that we will have to reproduce (because they are unextendable and unusable from outside the class), mock and string together the classes to ultimately get down to the socket class in java (which relies on a java native method -- which we would also need to reproduce and inject via mocks -- something I dont see frequently at that level).

The reason I am reaching out to Stack Overflow is to see how others are testing this/not testing this. I want to avoid testing this functionality in a performance environment at all costs. Another thought of mine was to set up a mockserver to respond to the httpclient with a programmable delay time. I haven't seen an example of that yet.

Answer 1

Taking the mock server route, I managed to set up a small web server with an API endpoint that I could test against. Here is the related code:

Lightweight server setup:

TJWSEmbeddedJaxrsServer server = new TJWSEmbeddedJaxrsServer();
server.setPort(SERVER_PORT);
server.getDeployment().getResources().add(new TestResource());
server.start();

API Endpoint:

/**
 * In order to test the timeout, the resource will be injected into an embedded server.
 * Each endpoint should have a unique use case.
 */
@Path("tests")
public class TestResource {

    @POST
    @Produces({MediaType.APPLICATION_XML})
    @Path("socket-timeout")
    public Response testSocketTimeout() throws InterruptedException {
        Thread.sleep(SOCKET_TIMEOUT_SLEEP);
        return Response.ok().build();
    }
}

Within the api endpoint related class, I can control the sleep timeout which then triggers a socket timeout within the httpclient class. Its a bit hacky, but it works to test the functionality in the way I wanted to (simple, lightweight and effective).