Search code examples
apache-kafkakafka-producer-api

Kafka Synchronous Producer


I come across the following like about synchronous send from the producer. I am aware of asynchronous mechanism in the context producer

Invoking get() on this future will block until the associated request completes and then return the metadata for the record or throw any exception that occurred while sending the record.

What is really mean by the associated request complete , I am quite this is not referring to the complete request , but to what extent this phrase is referring ? till the broker ? till the buffer used in the producer etc .. ?

How it is different when ack = all is used along with synchronous producer and async producer ? Both scenarios are blocked for acknowledgment ?


Solution

  • The producer has an internal queue to buffer some records (based on the configurations linger.ms and min.batch.size). There is an internal background thread that requests to the broker (where the leader partition is located) to send the batched data. Depending on the configuration max.connections.in.flight the producer can have multiple request in parallel. This is meant by "associated request".

    How it is different when ack = all is used along with synchronous producer and async producer ? Both scenarios are blocked for acknowledgment ?

    The get method will block until the request to the broker is completed, meaning it received all acknowledgments based on the acks setting.

    Using the get method always leads to a synchronous producer.