Cannot deserialize MessagePack back to JSON

Question

In a database column the bytes from a JSON serialized with the ygoe/msgpack.js JavaScript library are stored.

In C# I use the neuecc/MessagePack-CSharp library to deserialize this value into JSON:

//From database (bytes generated from JavaScript ygoe/msgpack.js)
var bytes = connection.Query<byte[]>(sql).FirstOrDefault(); //Size: 11160 - Get bytes from database
var hexa = System.Text.Encoding.UTF8.GetString(bytes);  //=> Value: 8ca76...
var hexaString = FromHexString(hexa);   //=> Value: ��meeting��id�\0\r�ۭmeetingNumber�2018194530...
var test1 = MessagePackSerializer.Deserialize<dynamic>(bytes);  //=> Value: 56
var test2 = MessagePackSerializer.Deserialize<RootObject>(bytes);  //=> System.InvalidOperationException: 'code is invalid. code:56 format:positive fixint'
var test3 = MessagePackSerializer.ToJson(bytes);    //=> Value:      "56"

//Manually
var jsonTest = "...JSON data..."; //Paste hexa on an online tool and use associated JSON (https://toolslick.com/conversion/data/messagepack-to-json#)
var test4 = MessagePackSerializer.Serialize(json);  //Size: 10893
var test5 = MessagePackSerializer.Deserialize<dynamic>(test4);  //OK
var test6 = MessagePackSerializer.Deserialize<RootObject>(test4);  //System.InvalidOperationException: 'code is invalid. code:218 format:str 16'

var test7 = MessagePackSerializer.FromJson(json);   //Size: 5928
var test8 = MessagePackSerializer.ToJson(test7);    //OK

As you can see above I tried several way to deserialize and none worked. I often get 56 or 218 in output but my JSON does not contain this value. I tried with all resolvers, none of them worked.

However, when copy/pasting the hex on this website I was able to get the JSON, meaning the bytes in the database seem valid.

Any idea what could be wrong?

Answer 1

The solution was to convert to HEX UTF8 and then convert back to byte[].

Code :

public RootObject Deserialize(byte[] bytes)
{
    var hexUtf8 = Encoding.UTF8.GetString(bytes);
    var bytesUtf8 = HexStringToByteArray(hexUtf8);

    var serializer = MessagePackSerializer.Get<RootObject>();
    var stream = new MemoryStream(bytesUtf8);
    return serializer.Unpack(stream);
}

public static byte[] HexStringToByteArray(this string hex)
{
    var result = new byte[hex.Length / 2];
    for (var i = 0; i < result.Length; i++)
    {
        result[i] = System.Convert.ToByte(hex.Substring(i * 2, 2), 16);
    }
    return result;
}