Unreliable response from java.net.http.HttpClient call using Java 11 and Spring Boot

Question

I'm writing a back-end application in Spring Boot that calls another API from a third party.

I'm having a problem with this particular call, which retrieves a token object containing a bearer token, which then I use in their other endpoints. The retrieved token sometimes works, most of the time it doesn't, when calling the other endpoints, resulting in an unauthorized response.

@RestController
public class CotizacionController {

    Logger logger = LoggerFactory.getLogger(CotizacionController.class);

    @Value("${service.credentials.tokenServer}")
    private String tokenServer;

    @Value("${service.credentials.grantType}")
    private String grantType;

    @Value("${service.credentials.username}")
    private String username;

    @Value("${service.credentials.password}")
    private String password;

    HttpClient client = HttpClient.newHttpClient();

    @RequestMapping("/create")
    public Object Create() throws IOException, InterruptedException {

        HashMap<String, String> parameters = new HashMap<>();
        parameters.put("grant_type", grantType);
        parameters.put("username", username);
        parameters.put("password", password);

        String form = parameters.keySet().stream()
                .map(key -> key + "=" 
                                + URLEncoder.encode(parameters.get(key), 
                                                    StandardCharsets.UTF_8))
                .collect(Collectors.joining("&"));

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(tokenServer))
                .header("Content-Type", "application/x-www-form-urlencoded")
                .POST(BodyPublishers.ofString(form)).build();

        HttpResponse<?> response = client.send(request, BodyHandlers.ofString());

        TokenResponse result = new ObjectMapper().readValue(response
                                   .body().toString(), TokenResponse.class);

        return result;
    }

}

And here's an example token object:

{
    "access_token": "z-bu-Pde6M2dlPiaRzd5XpTrT7ohpFQZe157HHVLfdKJWsdmKCloK7AYGEw7SLCe28tjYAxo8MZOE_3W00HEa-bqgUvcrAKfxIubAq0UGXv7jLPWbRwWzhAUCDon3kdstUrJ_OKRN2y26W6qyDBGDqlP5NRSF4unH_pD_ShmpDlSxZdYUqD0da5Y2_uO6YRs5GuWA7XhI9sPa98SxuXN_dwiDJVif418xK646fUgWR8",
    "token_type": "bearer",
    "expires_in": "3599"
}

Retrieving tokens using postman works perfectly fine, so it couldn't be an issue from the third party API. I also have this same service implemented in .NET Core 3 and it also works perfectly fine over there.

What confuses me the most is that the actual HttpClient call works, I do get a correct Json which is mapped to my TokenResponse object just fine. It's just that the token value is invalid... sometimes.

I've also tried using RestTemplate and WebClient Spring libraries, but the results are the same. Call works, but retrieved token is invalid.

At first I thought I was having a race condition, since initially I had another HttpClient in there with another endpoint using the response from the token call. So I simplified it into only the token call and manually copying the token value into postman requests. Didn't work.

Then I thought maybe my HttpClient authorization header was malformed, but that's disproven since simply copying the token into a protected endpoint using a postman request shows that the token doesn't work.

And other things I've tried:

• Pasting the form string I generate in the controller into the Postman request to make sure it's valid.
• Check that the URLEncoder is not messing up any form values.
• Copy the token value from the token object to use in another endpoint with Postman.
• Skip object mapping and return a simple String, and manually copying the token value from the response in Postman so I can then use it in another endpoint.

I'm pretty lost at this point, only thing that comes to mind is that maybe the HttpClient.send() method might be parsing the body in a way that might be affecting the content? I doubt it, but I don't see what else could be happening.

Answer 1

The solution was related to cookies!

The token server response was sending 2 set-cookie headers which in Postman and .NET Core were being automatically handled and set to subsequent HTTP requests. The 3rd party API was behind a load balancer and generating these session cookies.

I solved this by implementing a system-wide CookieHandler with the following code in my main method.

public static void main(String[] args) {
        CookieHandler.setDefault(new CookieManager());

        SpringApplication.run(Main.class, args);
    }

Then building my HttpClient object like this:

...
HttpClient client = HttpClient.newBuilder().cookieHandler(CookieHandler.getDefault()).build();
...

This way, the response set-cookie and request cookie headers will be handled automatically and work across all calls made by this HttpClient.

By default, CookieHandler is created with a CookiePolicy.ACCEPT_ORIGINAL_SERVER parameter. My understanding is that this makes cookies work only if they are set and requested by the same host. Check out the docs for more options on CookiePolicy