Search code examples
.netprotobuf-net

Protobuf Deserialisation

We are using protobuf for few years now and it great for what we do.

However, lately we are experiencing few de-serialisation issue.

Previously deserialisation was done serially, now to speed up the process we have used all cores system can provide.

Scenario:

  1. We are generating report, which includes 100 images.
  2. Each image is a widget and each widget has its own data source.
  3. We have more then one widget accessing same data(stored in protobuf).
  4. Attached is a screenshot, which highlights 2 widgets begin deserialised.
  5. Of them 1st is successfully deserialised.
  6. 2nd one fails deserialisation with below attached error message

Note: 1st and 2nd deserialisation happens a few milliseconds apart.

Error:

Context: 1 errors generating widgets. AnalysisTaskGlobalID: d549f367-f42f-45fa-b230-f2839a6fefe6 Error: Exception of type 'ProtoBuf.ProtoException' was thrown. Stack: at ProtoBuf.ProtoReader.StartSubItem(ProtoReader reader) in c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 599 at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type) in c:\Dev\protobuf-net\protobuf-net\ProtoReader.cs:line 531 at proto_2(Object , ProtoReader ) at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:line 506 at ProtoBuf.Serializer.Deserialize[T](Stream source) in c:\Dev\protobuf-net\protobuf-net\Serializer.cs:line 69

We don't think its a multi threading issue!, as we tried synchronizing de-serialisation process.

However, when adding a sleep of 1000ms before de-serialisation, works most of the time!

Any thoughts on this, please let me know.

Solution

  • This at in the end was not a protobuf error. Our application was not sufficiently synchronized.