Search code examples
c#.netserializationdeserializationprotobuf-net

protobuf-net deserializing: "Arithmetic operation resulted in an overflow."


I'm using protobuf-net for serializing/deserialzing my models.

My model is rather simple, serialization seems to work all the time, but if I add specific types to my model deserialization later seems to fail.

I get an "Arithmetic operation resulted in an Overflow" exception as soon as I add "int", "long" or "DateTime" to my models.

Model:

[ProtoContract]
public class MyModel
{
    [ProtoMember(1)]
    public DateTime Time { get; set; }

    [ProtoMember(2)]
    public List<string> SomeList { get; set; }

    [ProtoMember(3)]
    public string Key { get; set; }

    [ProtoMember(4)]
    public string Value { get; set; }
}

When I remove the "Time" Property it always seems to work.

Exception:

 at ProtoBuf.ProtoReader.TryReadUInt64VariantWithoutMoving(UInt64& value) in c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 375
   at ProtoBuf.ProtoReader.ReadInt64() in c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 357
   at ProtoBuf.BclHelpers.ReadTimeSpanTicks(ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\BclHelpers.cs:line 191
   at ProtoBuf.Serializers.DateTimeSerializer.Read(Object value, ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\Serializers\DateTimeSerializer.cs:line 35
   at ProtoBuf.Serializers.PropertyDecorator.Read(Object value, ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\Serializers\PropertyDecorator.cs:line 77
   at ProtoBuf.Serializers.TypeSerializer.Read(Object value, ProtoReader source) in c:\Dev\protobuf-net\protobuf-net\Serializers\TypeSerializer.cs:line 230
   at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type type, Object value, Boolean noAutoCreate) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 700
   at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 589
   at ProtoBuf.Serializer.Deserialize[T](Stream source) in c:\Dev\protobuf-net\protobuf-net\Serializer.cs:line 77

Am I doing something wrong?

[EDIT]

private static void Main(string[] args)
        {
            var proto = new SerializeProtoTest();
            var model = new MyModel
                            {
                                Key = "abc",
                                SomeList = new List<string> { "cde" },
                                Time = DateTime.UtcNow,
                                Value = "something"
                            };
            var s = proto.Serialize(model);
            var d = proto.Deserialize<MyModel>(s);
            Console.ReadKey();
        }

        [ProtoContract]
        public class MyModel
        {
            [ProtoMember(3)]
            public string Key { get; set; }

            [ProtoMember(2)]
            public List<string> SomeList { get; set; }

            [ProtoMember(1)]
            public DateTime Time { get; set; }

            [ProtoMember(4)]
            public string Value { get; set; }
        }

        public class SerializeProtoTest
        {
            public T Deserialize<T>(string value)
            {
                using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(value)))
                {
                    return Serializer.Deserialize<T>(ms);
                }
            }

            public string Serialize<T>(T obj)
            {
                using (var ms = new MemoryStream())
                {
                    Serializer.Serialize(ms, obj);
                    var buffer = ms.ToArray();
                    return Encoding.UTF8.GetString(buffer, 0, buffer.Length);
                }
            }
        }
    }

Solution

  • From comments:

    most likely : encoding inappropriately

    Called it! (from your edit)

    return Encoding.UTF8.GetString(buffer, 0, buffer.Length);

    protobuf data is not text. You cannot use a text encoding to get a text representation of it - in fact, you are using this text encoding backwards, and it has no defined behaviour here. You have corrupted the data. A larger write up (because I see this often) is here (the first section).

    However, the short version is: don't do that. If you need a string, use base-64 or similar instead:

    var buffer = ms.GetBuffer();
    return Convert.ToBase64String(buffer, 0, (int)ms.Length);
    

    (likewise, use Convert.FromBase64String to reverse this process)

    However, if possible it would be preferable to simply avoid the need to pass around string. byte[] would work fine (i.e. return ms.ToArray()).