Search code examples
apache-kafkakafka-producer-api

How Apache Kafka maintain order when it sends multiple messages to broker in parallel?


By default, max.in.flight.requests.per.connection == 5, means Kafka will send up to 5 messages to broker in parallel. How can it manage the order if those message go into the same partition in parallel?


Solution

  • Kafka cannot guarantee ordering of those messages if you set retries > 0.

    If you want to have ordering guarantees you need to set this value to max.in.flight.requests.per.connection == 1.

    The explanation is given in the Kafka doc on producer configs retries:

    Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change the ordering of records because if two batches are sent to a single partition, and the first fails and is retried but the second succeeds, then the records in the second batch may appear first.