My company is consider using protobuf3 as an message communication schema between our many microservices, and we struggle with find a solution for information lost when handling a message that transfer between more then 2 services and have more then one version, and I'll explain by writing an example:
I have three microservices, A,B,C that communicate with each other using the same schema:
message FooV1 {
int32 x = 1;
}
Where A send messages to B, B alter the messages and send the result to C (A->B->C). Now let's say I alter the schema and add another field:
message FooV2 {
int32 x = 1;
int32 y = 2;
}
Then, I update only services A and C with the new schema, and B will remain with the old schema, like this: A:V2, B:V1, C:V2.
Supposed A send x=1 and y=2 to B. B knows only tag 1, so he don't parse y at all. Then he alter x by changing it, for instance, to be 0. Then B send it to C. C now get x that equal to 0, but in the process y is 'vanished' and even though A and C both support the new schema, and B doesn't care about y, we still lose data in the process.
I want C to know y even thought B doesn't recognize y
Is there any way to tell protobuf that even thought the service won't parse an unknown tag he won't get rid of the data but append it back to the message on sending? If there is a way to do that, how does it work? Thanks.
I found this paragraph on google's protobuf3 documentation:
Originally, proto3 messages always discarded unknown fields during parsing, but in version 3.5 we reintroduced the preservation of unknown fields to match the proto2 behavior. In versions 3.5 and later, unknown fields are retained during parsing and included in the serialized output.
so using protobuf3.5+ was the answer for me