I'm using Protobuf for the communication between my web client and server (C#), using WebSocket. On the client, the de/serialization is done through Protobuf.js and, on the server, using protobuf-net.
The problem is that, when using aggregation with abstract classes, protobuf-net can't deserialize the data sent by Protobuf.js.
This is the stack trace:
ProtoException: No parameterless constructor found for Base.
at ProtoBuf.Meta.TypeModel.ThrowCannotCreateInstance(Type type) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 1397
at proto_6(Object , ProtoReader )
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Serializers\CompiledSerializer.cs:line 57
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Meta\RuntimeTypeModel.cs:line 775
at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type) na c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 579
at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader) na c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 566
at proto_2(Object , ProtoReader )
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Serializers\CompiledSerializer.cs:line 57
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Meta\RuntimeTypeModel.cs:line 775
at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type type, Object value, Boolean noAutoCreate) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 700
at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 589
at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 566
at ProtoBuf.Serializer.Deserialize[T](Stream source) na c:\Dev\protobuf-net\protobuf-net\Serializer.cs:line 77
at ProtobufPolymorphismTest.Program.Main(String[] args) na c:\Desenvolvimento\Testes\ProtobufPolymorphismTest\ProtobufPolymorphismTest\Program.cs:line 30
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
This is the contract:
[ProtoContract]
[ProtoInclude(100, typeof(Child))]
abstract class Base
{
[ProtoMember(1)]
public int BaseProperty { get; set; }
}
[ProtoContract]
class Child : Base
{
[ProtoMember(1)]
public float ChildProperty { get; set; }
}
[ProtoContract]
class Request
{
[ProtoMember(1)]
public Base Aggregate { get; set; }
}
And this is the code to reproduce the error. As the serialization is working, I'm only providing the result as a byte array. If it helps, I can provide the steps I took to get to the serialized values.
// This is the object serialized
Child child = new Child() { ChildProperty = 0.5f, BaseProperty = 10 };
Request request = new Request() { Aggregate = child };
// This is the byte representation generated by protobuf-net and Protobuf.js
byte[] protoNet = new byte[] { 10, 10, 162, 6, 5, 13, 0, 0, 0, 63, 8, 10 };
byte[] protoJS = new byte[] { 10, 10, 8, 10, 162, 6, 5, 13, 0, 0, 0, 63 };
// Try to deserialize the protobuf-net data
using (System.IO.MemoryStream ms = new System.IO.MemoryStream(protoNet))
{
request = Serializer.Deserialize<Request>(ms); // Success
}
// Try to deserialize the Protobuf.js data
using (System.IO.MemoryStream ms = new System.IO.MemoryStream(protoJS))
{
request = Serializer.Deserialize<Request>(ms); // ProtoException: No parameterless constructor found for Base.
}
If I add SkipConstructor = true on the Base class definition, the error changes to "MemberAccessException: Cannot create an abstract class" with the following stack trace. If I remove the abstract from the Base class definition, it works as expected.
System.MemberAccessException: Cannot create an abstract class.
at System.Runtime.Serialization.FormatterServices.nativeGetUninitializedObject(RuntimeType type)
at ProtoBuf.BclHelpers.GetUninitializedObject(Type type) na c:\Dev\protobuf-net\protobuf-net\BclHelpers.cs:line 38
at proto_6(Object , ProtoReader )
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Serializers\CompiledSerializer.cs:line 57
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Meta\RuntimeTypeModel.cs:line 775
at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type) na c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 579
at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader) na c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 566
at proto_2(Object , ProtoReader )
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Serializers\CompiledSerializer.cs:line 57
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source) na c:\Dev\protobuf-net\protobuf-net\Meta\RuntimeTypeModel.cs:line 775
at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type type, Object value, Boolean noAutoCreate) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 700
at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 589
at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type) na c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 566
at ProtoBuf.Serializer.Deserialize[T](Stream source) na c:\Dev\protobuf-net\protobuf-net\Serializer.cs:line 77
at ProtobufPolymorphismTest.Program.Main(String[] args) na c:\Desenvolvimento\Testes\ProtobufPolymorphismTest\ProtobufPolymorphismTest\Program.cs:line 30
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
I'm not sure why the binary representation generated through protobuf-net and Protobuf.js are different, but they both seem valid as it works if the Base class is not abstract.
Any ideas on why this is happening or a way to work around without removing the abstract from Base class?
Thanks in advance!
UPDATE
This is the code I used to generate the byte serialization through Protobuf.js:
<script src="//raw.githubusercontent.com/dcodeIO/ByteBuffer.js/master/dist/ByteBufferAB.min.js"></script>
<script src="//cdn.rawgit.com/dcodeIO/ProtoBuf.js/master/dist/ProtoBuf.js"></script>
<script type="text/javascript">
// Proto file
var proto = "";
proto += "package ProtobufPolymorphismTest;\r\n\r\n";
proto += "message Base {\r\n";
proto += " optional int32 BaseProperty = 1 [default = 0];\r\n";
proto += " // the following represent sub-types; at most 1 should have a value\r\n";
proto += " optional Child Child = 100;\r\n";
proto += "}\r\n\r\n";
proto += "message Child {\r\n";
proto += " optional float ChildProperty = 1 [default = 0];\r\n";
proto += "}\r\n\r\n";
proto += "message Request {\r\n";
proto += " optional Base Aggregate = 1;\r\n";
proto += "}";
// Build the entities
var protoFile = dcodeIO.ProtoBuf.loadProto(proto);
var requestClass = protoFile.build("ProtobufPolymorphismTest.Request");
var baseClass = protoFile.build("ProtobufPolymorphismTest.Base");
var childClass = protoFile.build("ProtobufPolymorphismTest.Child");
// Build the request
var base = new baseClass();
base.BaseProperty = 10;
base.Child = new childClass();
base.Child.ChildProperty = 0.5;
var request = new requestClass();
request.Aggregate = base;
// Serialize
var bytes = new Uint8Array(request.toArrayBuffer());
var str = "new byte[] { " + bytes.join(", ") + " };";
console.log(str);
</script>
WORKAROUND
As Marc explained, protobuf-net doesn't support polymorphism when the field order is inverted. As a workaround, specific to Protobuf.js, you can change the order of the fields in the .proto file for it to serialize in the correct order.
In my case, changing the .proto file to the following solved the problem:
package ProtobufPolymorphismTest;
message Base {
// the following represent sub-types; at most 1 should have a value
optional Child Child = 100;
optional int32 BaseProperty = 1 [default = 0];
}
message Child {
optional float ChildProperty = 1 [default = 0];
}
message Request {
optional Base Aggregate = 1;
}
(Note the optional Child Child = 100; before the BaseProperty)
The long and short of it is that protobuf-net's polymorpism support expects the sub-type to be first in the message (or more specifically: for any object, the type will be fixed before data is provided). In the js output, the field data for BaseProperty comes first - perfectly reasonably, perhaps. But since there is no over-arching protocol definition of how inheritance should behave, protobuf-net's implementation was only really ever intended to work with itself. In terms of the bytes, this actually comes down to where the field marker "162, 6" (and the associated length/data, "5, 13, 0, 0, 0, 63") appears.
The library could potentially be reworked to allow any field order for polymorphism, but: it would take some effort. I am aware that it is usually expected to process fields in any order, but since this is already outside of the specification, I didn't focus on this. All other data fields are accepted in any order - only polymorphism works this way.
In the general case: since polymorphism is not part of the specification, I would strongly recommend avoiding polymorphism when working between libraries.
Note: you can probably force this to work by ensuring the polymorphism fields are lower (numerically) than the data fields.