Can simple protobuf types be migrated to "optional"

Question

In protobuf version 3 required and optional keywords first have been removed, since required often caused problems protobuf issue 2497.

Recently the 'optional' keyword has been reintroduced protobuf v3.15.0.

Is it possible to simply add the optional keyword to an existing message?

I.e. change

message Test {
    int32 int32_value = 1;
    string text_value = 2;
}

to

message Test {
    optional int32 int32_value = 1;
    optional string text_value = 2;
}

Or will this break the binary format?

Answer 1

non-optional primitive types in protobuf don't accept null-values and normally also map to non-nullable types like int in Java or C#. But this doesn't mean, that the field is always included in the binary representation.

In fact, if a field contains the default value for the corresponding type the field is omitted in the binary representation.

Thus the following message

message Test {
    int32 int32_value = 1;
    string text_value = 2;
}

Test test = new Test();
byte[] buffer = test.ToByteArray();

gets serialized to buffer containing an empty byte[]. So missing fields default to default values without the use of optional.

If the optional keyword is changing the behaviour for missing fields in the binary format and for default values specified: Missing fields indicate the field has not been specified and indicate null. Setting default values will not result in an empty byte[] but in the default values being serialized.

Thus changing a primitive field to optional won't break the format, but will change the semantics:

All fields of old messages that have been specified with the default value will be interpreted as null. Other values are not affected.

The same for optional being removed from a field: The api won't break, but change semantics. Unspecified fields will then default to default values for the corresponding type.