What's the difference between Protobuf ToByteArray and WriteDelimitedTo function in C#?

Question

So I'm new to C# and I need to communicate with server (implemented in go) using protobuf + websocket. My initial implementation looks like this:

var command = new Command
            {
                //some fields
            }.ToByteArray();
_websocketClient.SendAsync(command);

where _websocketClient is using websocket-sharp client.

However, on server side I cannot unmarshall the bytes from request and got error

proto: Command: illegal tag 0 (wire type 1)

This is the error from proto generated code. It looks like the bytes I send is somehow invalid. I then switched to this implementation

using var ms = new MemoryStream();
command.WriteDelimitedTo(ms);
_websocketClient.SendAsync(ms.GetBuffer());

and everything is working fine now. So it looks like this is caused by the way that struct is serialized. But I didn't find any documentation explaining what writeDelimitedTo actually does and how it is different from ToByteArray().

Can someone explain to me what's the difference here that caused my first implementation to fail? Much appreciated and many thanks!

Answer 1

So I've dug into the source code, and the answer here is that indeed they write different things:

• ToByteArray writes the regular protobuf payload of the message, via WriteTo
• WriteDelimitedTo does the same payload, but with an additional length prefix

The purpose of the latter in the general case is because protobuf is not a self-terminating message format, which means: it doesn't know when any single message stops - you just keep reading until you reach an EOF. This is problematic if you are writing multiple messages to a single stream, for example a socket server - hence you need "framing" to break this into a series of logical payloads, and WriteDelimitedTo is a simple mechanism for providing this "framing".

In the case of web-sockets, web-sockets itself provides framing, so in theory you don't actually need to prefixed form, however when it comes to serialization, the more important question is:

What does the other end expect?

So ultimately: figure out which of those two alternative layouts is expected by the Go server, and use that - even if it means it is partly redundant in a framed protocol such as web-sockets. If you use the wrong alternative layout, it will be misinterpreted by the deserializer, leading to this kind of problem.

Technically ToByteArray is the more semantically correct (in a framed context), but: being "correct" doesn't help us if the receiver expects the prefixed form.