Does protobuf-net support ArraySegment (yet?)

Question

I have read Protobuf-net memcache provider fails on append/set ArraySegment which says it may be supported at some point.

I have also tried the test suggested in that question, which indicates that version 2.0.0.668 does not support it.

Has anyone had success using ArraySegments with protobuf-net, or have an idea for an efficient method of sending a byte array over the wire.

I.e. I have an object like this:

[ProtoContract] 
class Foo
{
    [ProtoMember(1)]
    public byte[] Data { get; set; }
}

And I want to serialize a different object T  into a byte array, assign it to Data, and then serialize Foo into another byte[]. (Sounds inefficent, but the Foo class must be agnostic to the type, could it be generic?).

The key thing is I would prefer to minimise allocation/GC, so I want to minimise the copying of arrays or allocation of new ones each time I serialize the data.

Here is some example code which demonstrates what I am trying to achieve more efficently:

public class DataSender<T>
{
    private ISerializer serializer; //Could be any kind of binary serializer 
    public void Send(T obj)
    {
        MemoryStream serializationBuffer = new MemoryStream(); // Inefficent allocation
        serializer.Serialize(serializationBuffer, obj);
        var sendable = new Foo(){ Data=serializationBuffer.ToArray() } // Inefficent copy
        Sender.Send(sendable);
     }
}

Any suggestions for replacements for my Foo object and/or sending code would be very welcome.

Answer 1

That looks like overly complicated to me. I think that your idea to make Foo generic is pretty good, so you could have:

[ProtoContract]
class Foo<T>
{
    [ProtoMember(1)]
    public T Data { get; set; }
}

And then you would instantiate a new Object with simple code like:

var foo = new Foo<Bar>() { Data = new Bar() { Data = "Some Data" } };

However,based on your additional comments you can't guarantee the object is serializable, or has proto attributes. For example the bar class could look like:

class Bar
{
    public string Data { get; set; }
}

In that case you can use the RuntimeTypeModel class to configure serialization at runtime.

Here is an example of full serialization and deserialization using protobuf with a dynamic runtime configuration for the data object:

using (var serializationBuffer = new MemoryStream())
{
    var foo = new Foo<Bar>() { Data = new Bar() { Data = "Some Data" } };

    RuntimeTypeModel.Default.Add(foo.Data.GetType(), false)
        .Add(foo.GetType().GetProperties().Select(p => p.Name).ToArray());

    Serializer.Serialize(serializationBuffer, foo);

    serializationBuffer.Seek(0, SeekOrigin.Begin);

    var deserialized = Serializer.Deserialize<Foo<Bar>>(serializationBuffer);
}

This way you can use any object as your data object, even if it doesn't have serialization attributes. However, you will have a performance penalty, since reflection is used to discover the type properties. However, you should get the flexibility you need.

The correct way to send this object over the wire using WCF would be to configure a WCF custom behavior using the protobuf ProtoEndpointBehavior and then WCF would automatically serialize and deserialize your Foo objects using protobuf. This means that the client would just make sure it is using objects decorated with the proto attributes, and send them over the wire. WCF would take care of the serialization/deserialization.

Here is a comprehensive example how to consume WCF protobuf services:

http://www.drdobbs.com/windows/working-with-protobuf-wcf-services/240159282?pgno=1

The efficiency of sending data over the wire depends on many factors. If you are sending big objects through the wire you can set WCF TransferMode to use Streaming, and then read the data in byte array chunks on the other end.

However, efficiency is something you should measure, not assume.