kafka does not have messages that producer sent

Question

I use spring framework and kafka that has 3 brokers clustered. I found out that consumer did not consume some messages (let say 0.01 percent between all sent messages) so in producer code I log message offset that returned by api:

ListenableFuture<SendResult<String, Message>> future = messageTemplate.sendDefault(id, message);
SendResult<String, Message> sendResult = future.get();
String offset = sendResult.getRecordMetadata().offset();

I use return offset to query kafka topic in all partition but it do not find the message (I test other offsets related to messages that consumers used and they are in kafka), whats the problem and how can I insure that message sent to kafka??

I also used messageTemplate.flush(); in producer

Answer 1

I find out that when topic leader of Kafka broker goes down Kafka will rebalance itself and another broker becomes the leader of that partition and if the ack config is not set to all there is a chance of losing some data in this procedure. so change the config to

ack=all

also, there is a chance of losing data if minimum in sync replica becomes less than 2, so set it at least to 2.

min.insync.replicas = 2