Search code examples
javacachingapache-kafkaavroconfluent-schema-registry

How local cache is managed by Confluent Schema Registry

We plan to push millions of messages through Kafka, using the Schema Registry (SR) (Confluent) to not sent the full schema all the times.

The architecture is related in this picture: enter image description here

The fact is we want to avoid to have one call to the SR all the time the Producer want to send a message or the Consumer will receive one.

So I want to talk here about the "local" cache managed by the AvroSerializer and AvroDeserializer. I presume that contributors that has implemented this should have done well this job.

But due to the amount of implicated calls, I just want to be sure what I written in red in the above schema

I didn't find any functions to manage this "local" cache, do you know something to check its content, size ...

Thanks a lot!

Solution

  • Ok finally I tested it myself seeting in place this architecture : 2 VMBox under ubuntu, one for producer/broker/consumer and another for the schema registry (confluent).

    I set also a WireShark TCP packets analyser in bot VM.

    When I launch my producer for one message or for thousands of messages, we have the same count of packets exhanged :

    enter image description here

    Setting some sleep (in Java) I can see that the call to the Schema registry is done only when the first message has to be sent, not when the producer is created.

    Hoping this can help...