how to handle "Transfer-Encoding=chunked" in SI HttpRequestExecutingMessageHandler

Question

I am calling an external server using HttpRequestExecutingMessageHandler. I am using a JSON to object transformer to convert the JSON data. But I am getting the following exception.

Caused by: com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens at [Source: (String)"�

when I checked the headers I found the following. Transfer-Encoding=chunked

Is this the reason for the exception log?

The outbound gateway and JsonToObjectTransformer are given below:

    @ServiceActivator(inputChannel = "channelOutboundRequest")
    @Bean
    public HttpRequestExecutingMessageHandler outboundGateway() {
        final HttpRequestExecutingMessageHandler handler = new HttpRequestExecutingMessageHandler(
                endpoint);
        handler.setExpectedResponseType(String.class);
        handler.setHttpMethod(HttpMethod.POST);
        handler.setOutputChannelName("channelResponse");
        handler.setAdviceChain(Collections.singletonList(advice()));
        return handler;
    }
    @Bean
    @Transformer(inputChannel = "channelResponse", outputChannel = "channelReply")
    public JsonToObjectTransformer transformer(ObjectMapper objectMapper) {
        final JsonObjectMapper<?, ?> mapper = new Jackson2JsonObjectMapper(objectMapper);
        return new JsonToObjectTransformer(DetailsDTO.class, mapper);
    }

If the header is causing the issue, how can I handle the response?

Note: If I hit the external server directly using postman, I am getting the response in JSON structure.

I have no idea what's wrong here.. If I use the simple restemplate call like below, it works properly.

JSONObject jsonObject = new JSONObject("{\"code\":\"F001\",\"transactionId\":\"1008566223232\"}");
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        headers.setBearerAuth("token");
        HttpEntity<String> request = 
                  new HttpEntity<String>(jsonObject.toString(), headers);
        String respns = restTemplt.postForObject("http://endpoint", request, String.class);
        System.out.println(respns);
        JSONObject response = new JSONObject(respns);

One difference I could find was in the response headers. those are given below:

Outbound gateway response headers :- {Transfer-Encoding=chunked, http_requestMethod=GET, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel@d8b195e, Server=nginx, Accept=/, Connection=keep-alive, User-Agent=PostmanRuntime/7.28.0, Host=localhost:8901, Accept-Encoding=gzip, deflate, br, http_statusCode=200 OK, Date=1622533072000, Authorization=Bearer token, replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel@d8b195e, Cache-Control=no-cache, ETag=W/"1009-5SzdL+uWyY6ZcMWht5dMtm2Sxlc", Content-Encoding=gzip, http_requestUrl=http://inboundurl, id=be07fc8d-d478-5fa9-33e4-61a2b5f92468, Content-Length=207, contentType=application/json;charset=utf-8, Content-Type=application/json, requestFrom=CUSTOM_HEADER, timestamp=1622533092827}

Normal restTemplate call response header [Server:"nginx", Date:"Tue, 01 Jun 2021 07:34:54 GMT", Content-Type:"application/json; charset=utf-8", Content-Length:"4105", Connection:"keep-alive", Access-Control-Allow-Origin:"*", Content-Security-Policy:"default-src 'self';base-uri 'self';block-all-mixed-content;font-src 'self' https: data:;frame-ancestors 'self';img-src 'self' data:;object-src 'none';script-src 'self';script-src-attr 'none';style-src 'self' https: 'unsafe-inline';upgrade-insecure-requests", X-DNS-Prefetch-Control:"off", Expect-CT:"max-age=0", X-Frame-Options:"SAMEORIGIN", Strict-Transport-Security:"max-age=15552000; includeSubDomains", X-Download-Options:"noopen", X-Content-Type-Options:"nosniff", X-Permitted-Cross-Domain-Policies:"none", Referrer-Policy:"no-referrer", X-XSS-Protection:"0", ETag:"W/"1009-llD9DqxYkEsjyikWajYk+16cb1k""]

Can anyone please help?

Answer 1

After many trials and errors, I found the reason for this. Accept-Encoding=gzip, deflate, br this header in the outbound gateway request is the root cause. the response I am getting is a long string and it is getting compressed because of this header. i added a header filter before the outbound gateway to remove this header.

    @Bean
    @Transformer(inputChannel = "channelHeaderFilterReq", outputChannel = "channelHeaderFilterRes")
    public HeaderFilter filter() {
        return new HeaderFilter("Accept-Encoding");
    }

Now everything works fine..!!