ApacheCamel exception while processing UTF char to JSON. (Invalid UTF-8 middle byte)

Question

We have a Camel gateway service which

• receives a request in JSON, transform it to SOAP request & calls a SOAP service.

• And on receiving the SOAP response transform it to JSON & sent it back to the caller service.

Now while transforming the SOAP response to POJO to JSON somewhere it is throwing an exception as below,

com.ctc.wstx.exc.WstxLazyException: [com.ctc.wstx.exc.WstxLazyException] Invalid UTF-8 middle byte 
0x26 (at char #8412, byte #7999)

SOAP response is coming with some UTF chars like “ â “ , which is causing the problem. How to handle or ignore these UTF chars & transform to JSON ?

Camel code : Routes.java

private void getPlanDetailsRequestRoute(JacksonDataFormat jacksonDataformat, SoapJaxbDataFormat soapJaxbDataFormat, DataFormat jaxb) {
        from("servlet:/PlanDetailsRequestService?matchOnUriPrefix=true").unmarshal(jacksonDataformat).to("dozer:transformOrder?mappingFile=DozerMapping.xml&sourceModel=com.ong.vx.gatewayservice.dto.PlanRecord&targetModel=com.ong.vx.gatewayservice.wsdl.GetPlanDetailsRequest").marshal(soapJaxbDataFormat)
                .to("bean:SoapEnvelopeWrapper?method=process").to(WS_URI).choice()
                .when(header(Exchange.HTTP_RESPONSE_CODE).isEqualTo(500)).to("bean:ExceptionExtracter?method=process")
                .endChoice().otherwise().unmarshal(soapJaxbDataFormat).end().unmarshal(jaxb).marshal().json(JsonLibrary.Jackson)
                .setHeader(Exchange.CONTENT_TYPE, constant("application/json"));
    }

Exception :

com.ctc.wstx.exc.WstxLazyException: [com.ctc.wstx.exc.WstxLazyException] Invalid UTF-8 middle byte 0x26 (at char #8412, byte #7999)
at com.ctc.wstx.exc.WstxLazyException.throwLazily(WstxLazyException.java:40)
at com.ctc.wstx.sr.StreamScanner.throwLazyError(StreamScanner.java:724)
at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3758)
at com.ctc.wstx.sr.BasicStreamReader.getTextCharacters(BasicStreamReader.java:914)
at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.handleCharacters(StAXStreamConnector.java:327)
at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:191)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:415)
at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:386)
at org.apache.camel.converter.jaxb.JaxbDataFormat.unmarshal(JaxbDataFormat.java:255)
at org.apache.camel.dataformat.soap.SoapJaxbDataFormat.unmarshal(SoapJaxbDataFormat.java:284)
at org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:69)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.ChoiceProcessor.process(ChoiceProcessor.java:117)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:542)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:120)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:197)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97)
at org.apache.camel.http.common.CamelServlet.doService(CamelServlet.java:195)
at org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:74)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:105)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at 
org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
atorg.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at 
org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 middle byte 0x26 (at char #8412, byte #7999)
at com.ctc.wstx.sr.StreamScanner.constructFromIOE(StreamScanner.java:640)
at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:1004)
at com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4729)
at com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3802)
at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3756)
... 71 common frames omitted
Caused by: java.io.CharConversionException: Invalid UTF-8 middle byte 0x26 (at char #8412, byte #7999)
at com.ctc.wstx.io.UTF8Reader.reportInvalidOther(UTF8Reader.java:315)
at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:206)
at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:89)
at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
at com.ctc.wstx.sr.StreamScanner.loadMore(StreamScanner.java:998)
... 74 common frames omitted

Answer 1

The exception you receive sound like UTF-8 characters of the SOAP response are encoded as ISO-8859-1 instead of UTF-8. See also this answer for details.

Have you checked that the response is correctly encoded (according to the XML declaration?) and the charset header of the response is correct? If not, it would be best to fix the SOAP service (the source).

If you have no control over the source, your best bet is probably to do a hacky workaround. For example using a regex to replace the distracting characters with the correct characters before you transform it.