We have microservice architecture on our project. There is a microservice where scheduler trigger logic of sending message to Kafka. The problem is that if there are more than one microservice instances, then each of them will send a message to Kafka. How can achieve idempotency between all producers? I mean that if 'equal' messages will be sent to Kafka with various producers, when it will be committed in log only once.
You cannot do that. Producers are independent.