Search code examples
goproto

golang protobuf marshal empty struct with fixed size


I hava a protobuf struct Data

in .proto:

message Data {
    uint64 ID = 1;
    uint32 GUID = 2;
}

in golang

b, err := proto.Marshal(&pb.Data{})
if err != nil {
    panic(err)
}
fmt.Println(len(b))

I got 0 length!

How can I make proto.Marshal always return fixed size no matter what pb.Data is?

ps.

pb.Data only contains int64 and int32


Solution

  • There are two issues here

    1) protobuf uses varint encoding for integers, so the size depends on the value, see this link

    2) zero value fields are not transmitted by default, so because the two integers are zero, even their field identifiers are not sent. I'm actually not sure there's even an option to send zero values looking at the docs

    if you set them both to 1, you will have more than zero bytes, but it still won't be fixed in length, depending on range of the values

    so, there's no real way to enforce fixed size in protobuf messages in general

    if you want fixed length messages, you are probably better off using direct structs-on-the-wire type encoding, but then that's harder for language interop as they'd all have to define the same message and you'd lose easy message migrations and all the cool stuff that protobuf gives.

    Cap'n Proto might have an option for fixed size structs, but they also generally compress which will, once again, produce variable length messages.

    If you describe the problem you are trying to ultimately solve, we may be able to suggest other alternatives.