Search code examples
apache-kafkaavroconfluent-schema-registry

Generate AVRO POJO using a schema in schema registry directly


In a confluent doc I see it mentions "It is useful to think about schemas as APIs". But when generating POJOs from schema, it refers a local schema. If the schema in the schema registry changes, according to my understanding the local schema should also needs to be updated, which is a maintenance overhead from my point of view.

Is there a way/pattern/plugin to generate the POJOs by referring the schema in the schema registry, using the schema registry APIs? So we do not need to keep schemas locally.


Solution

  • A Producer (or external process tightly coupled with a producer) is generally the one that initiates any schema change; there is no clear reason why a consumer would change a schema.

    If you need a SpecificRecord for a consumer, you can combine the schema-registry-maven-plugin download goal with the avro-maven-plugin schema goal. Otherwise, consumers can read GenericRecord without compiling any schemas.

    However, the "better" practice I've seen (for Java clients) is that they define the schema, it gets generated, then bundled and packaged as a standalone Maven dependency (separate module), then uploaded to Artifactory/Nexus. Only when the producers send events (if they do, at all), then the Schema Registry be updated, but if no messages are sent, then the schema is still available elsewhere.

    But yes, there is no great way around dependency management. The same could be said for any library, though, where you need to update some internal field/method.