Search code examples
c#unity-game-engineprotobuf-netrijndaelmanaged

unable to de-serialize data


I started created a Unity game as a Visual Studio project. I created a side project (WinForms) for testing as I don't know unit testing. There, all of my code works perfectly, the serialization class works flawlessly even when the protobuf-net DLL used is the one for Unity. Now, when I started to work on Unity, I copied over all of my code and DLLs for protobuf-net and MySQL.Data. For some reason, I cannot get the (basically)same code to work. I always get an exception when de-serializing, mainly this:

Invalid field in source data: 0

And this

Unexpected end-group in source data; this usually means the source data is corrupt

This is the stack trace for the last exception:

ProtoBuf.ProtoException: Unexpected end-group in source data; this usually means the source data is corrupt at ProtoBuf.ProtoReader.ReadFieldHeader () at (wrapper dynamic-method) NeohumanSoftware.AoN.UserInformation.User.proto_2 (object,ProtoBuf.ProtoReader) at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) at ProtoBuf.Meta.TypeModel.DeserializeCore (ProtoBuf.ProtoReader,System.Type,object,bool) at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type,ProtoBuf.SerializationContext) at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type) at ProtoBuf.Serializer.Deserialize (System.IO.Stream) <0x000ae> at NeohumanSoftware.AoN.Storage.SerializationHandler.DeSerialize (string) <0x00140> at NeohumanSoftware.AoN.Storage.DataHandler.DeserializeUser (string)

I omitted the last part of the stack trace as it's not relevant. I do not understand why it works in Visual Studio but not in Unity. This is the serializer:

using (MemoryStream ms = new MemoryStream())
{
    Serializer.Serialize(ms, ToSerialize);
    return ms.ToArray();
}

And this is the de-serializer:

using (MemoryStream ms = new MemoryStream(ToDeSerialize))
{
    return Serializer.Deserialize<T>(ms);
}

And this is the object to serialize which is the "heart" of the game as everything comes from User.user:

[ProtoContract]
internal class User
{
    internal static User user { get; set; }

    [ProtoMember(1)]
    internal Something uSomething { get; private set; }
    [ProtoMember(2)]
    internal bool uSomething2 { get; set; }
    [ProtoMember(3)]
    internal Something3 uSomething3 { get; set; }
    [ProtoMember(4)]
    internal Something4 uSomething4 { get; private set; }
    [ProtoMember(5)]
    internal Something5 uSomething5 { get; private set; }

All of these custom classes have the [ProtoContract] attribute along with at least one [ProtoMember], and I do not use any other attribute apart from these 2. This is the flow of the game:

  1. Data is created.
  2. Data is serialized.
  3. Serialized data is passed on for encryption (RijndaelManaged).
  4. Encrypted and serialized data is written to a file.
  5. Game ends
  6. Data is loaded from the file.
  7. Data is unencrypted using the exact same Key and IV (randomly generated and stored)
  8. Data is de-serialized.
  9. Game starts

EDIT: Trying to use Visual Studio's project to de-serialize Unity's data does NOT work. Trying to use Unity to de-serialize Visual Studio's data does NOT work. It seems like Unity does not like protobuf-net at all...

EDIT2: I changed the serializer method to:

byte[] b;
using (MemoryStream ms = new MemoryStream())
{
    Serializer.Serialize(ms, toXML);
    b = new byte[ms.Position];
    var fullB = ms.GetBuffer();
    Array.Copy(fullB, b, b.Length);
}

And this is what I got:

ProtoBuf.ProtoException: Invalid wire-type; this usually means you have over-written a file without truncating or setting the length; see Using Protobuf-net, I suddenly got an exception about an unknown wire-type at ProtoBuf.ProtoReader.SkipField () at (wrapper dynamic-method) NeohumanSoftware.AoN.UserInformation.User.proto_2 (object,ProtoBuf.ProtoReader) at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) at ProtoBuf.Meta.TypeModel.DeserializeCore (ProtoBuf.ProtoReader,System.Type,object,bool) at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type,ProtoBuf.SerializationContext) at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type) at ProtoBuf.Serializer.Deserialize (System.IO.Stream) <0x000ae>

If it helps, these are the read and write to file methods:

internal static void SaveUser(string userSerialized)
{
    File.WriteAllText(SomePath, userSerialized);
}

internal static void LoadUser(string userSerialized)
{
    File.ReadAllText(SomePath);
}

EDIT 3: In Unity's project, I had these that I did not have in Visual Studio's project:

internal static Coordinates FromVector3(Vector3 vector)
{
    return new Coordinates(vector.x, vector.y, vector.z);
}
internal static Vector3 ToVector3(Coordinates c)
{
    return new Vector3(c.X, c.Y, c.Z);
}

These are in the Coordinates class, which has [ProtoContract] and [ProtoMember] for X, Y and Z. Commenting these 2 methods out stops me from receiving exceptions but the serialization still doesn't work (the User properties are all "new")


Solution

  • SOLVED

    After giving up all hope on protobuf-net I switched over to DataCotractSerializer which somehow worked on Unity. I still got many issues so I decided to turn off the encryption and read the XML using Visual Studio; the file was perfectly ok. I still had a lot of issues with my 3 event handlers (2 were hooked up on game start, the other each time a special object is created), so I had to remove the event handlers and do the operations manually (what a pain in the a$$). After all of this, I managed to add Protobuf-net back and everything work as it should (note that protobuf-net gives me a 89 bytes file agains DCS' 3.5 KB file)

    So, the final solution to this extremely weird issue was:

    1. Disable a fully functional RijndaelManaged implementation
    2. Disable all event handlers

    If someone can help me understand why RijdnaelManaged did work on Visual Studio but not Unity, please leave a comment.