Wrong encoding of google cloud translate and Java

Question

I'm trying to use Google cloud translate. I think the problem is that Google cloud translate use UTF8 and the jvm use UTF16. So i got some typo in translations. For instance :

      public static void main(String... args) throws Exception {
    // Instantiates a client
    Translate translate = TranslateOptions.getDefaultInstance().getService();

    // The text to translate
    String text = "Bonjour, à qui dois-je répondre? Non, C'est l'inverse...";

    // Translates some text into Russian
    Translation translation =
        translate.translate(
            text,
            TranslateOption.sourceLanguage("fr"),
            TranslateOption.targetLanguage("en"));


    System.out.printf("Text: %s%n", text);
    System.out.printf("Translation: %s%n", StringEscapeUtils.unescapeHtml(translation.getTranslatedText()));
  }

will return :

"Translation: Hello, who should I answer? No, it's the opposite ..."

instead of :

Translation: Hello, who should I answer? No, it's the opposite ...

We can't change the encoding of a java String, and the Google Cloud Api will not accept anything (Byte[]?) but String.

Do someone know how to fix it?

Thank you for reading

Edit : This code is now working, I added the StringEscapeUtils.unescapeHtml from commons.apache dependencies. I do not know if there is an other way to do it.

Answer 1

It's not a problem of UTF8 / UTF16.
The answer of google is html encoded.

https://en.wikipedia.org/wiki/Unicode_and_HTML

This is common if you want to transmit unicode character using only ASCII in a xml/html context .