Can I change a (Netflix) Hollow model schema's field inlining once it's in use?

Question

I'm working on software that propagates, to a fleet of servers, a medium-sized data set that doesn't change very often. I'm using Netflix's Hollow library to do it.

I've been thinking of the lifecycle of my application, and how it will grow and change over time. I've looked at the notes on backwards compatibility in the Hollow docs, and I don't think my question is addressed there.

When tuning my model for speed and memory efficiency, I can foresee deciding that a field I was referencing by id would be better off inlined, or vice versa.

Is inlining backwards compatible? That is, if I have inlined a field:

@HollowInline String title;

and I determine that I'd be better served by making the field a reference instead, can I just remove the inline annotation?

String title;

Or, as seems more likely, will that break my clients?

My hypothesis is that the only safe way to change inlining status is to add a new field with the desired inlining status, migrate my producer and consumers to use it, and then remove the old one.

Answer 1

It is a backwards incompatible change as you surmise. Given this schema:

java class A { int a1; @HollowInline Integer a2; }

Changing int a1 to Integer a1 and removing the @HollowInline annotation are equivalent. Changing an existing field from inline to reference or reference to inline is a breaking schema change.

A couple strategies can work around this:

Strategy 1

1. introduce a new field name; in your example you could call it String titleRef or String titleV2
2. populate title and titleV2 with the same value
3. release a new client API
4. once all consuming apps/services have upgraded and switched to call getTitleV2() you can remove title from the schema and begin publishing data without it

Strategy 2

1. Given: you use a namespace or naming scheme (such as naming the path to your blobs a certain way and announcing your versions under a topic name)
2. make the breaking change to your schema
3. change or rev your namespace
4. stand up a new producer publishing and announcing to the new namespace running in parallel to the old producer
5. release a new client; consumers picking up this new client should be configured to use the new namespace (it could be baked in to a wrapper you release with the client, for instance)
6. once all consuming app/services are using new client with new namespace, retire old producer