Japanese character in AdaptiveCard Bot Framework V4

Question

I have been trying to print a simple card with a Japanese character but it keeps displaying boxes and unknown characters.

This is how I create my adaptive card, then I get the params and data in a json, just to make it neat.

        string[] paths = { ".", "Cards", "pickLanguageCard.json" };
        string fullPath = Path.Combine(paths);
        var adaptiveCard = File.ReadAllText(fullPath);
        return new Attachment()
        {
            ContentType = "application/vnd.microsoft.card.adaptive",
            Content = JsonConvert.DeserializeObject(adaptiveCard),
        };

Picture of the printed output:

As you can see the returned JSON data is also wrong. So this pinned it down to the main source of the Bot. I tried to the Json file containing the Japanese character, also change the encoding at the web.config but it didn't solve my problem. Back in Bot Framework v3, there is no problem in printing/displaying Japanese character. But when I tried v4 the Japanese characters get like that.

Any fix, solution, workaround will be appreciated. Thanks

Edit: Tried using encoding param in ReadAllText, (Encoding.UTF8, Encoding.UTF32, Encoding.Unicode). In UTF8, other japanese character get to print but destroy the format of the Json unable to parse, also it occurs in utf32 and unicode. In default the character is the same.

Edit: So after researching relentlessly, I found out that the JSON only encodes data to standard UTF-8 to make it lighter, and tried to using converting the characters to UTF-16 and it print successfully, but that seems wrong for me. Is there any other way to print correctly the Japanese characters?

Answer 1

When you edit JSON in Visual Studio 2019 and try to save the file with Japanese characters, Visual Studio will automatically offer to fix the format for you:

If you want to manually save your file with a specific encoding instead of relying on an automatic dialog box, you can use the Save with Encoding... option in the File > Save As... dialog:

If you select Codepage 65001, which is Unicode (UTF-8) with or without signature, your Japanese characters should display correctly: