How to notify user through kafka producer to consumer process

Question

I have a REST API which gets data from user and push this data to kafka topic using producer API, now a consumer process is running to consume messages from this topic. I need to send the status of consumer process to the API for every message. Is there any way to achieve this?

Answer 1

Apache Kafka is a message queue that decouples the producer side from the consumer side. It does not provide a mechanism for consumers to directly interact with producers.

That said, you have a few options. The most common is simply to make the processing side produce a message to another topic that is consumed by the "producer side". That way you have your data flowing in one direction and status updates in the other.

Another option is to have the consumer side call a REST API endpoint on the producer side to update its status. This is a bit less elegant but could work depending on your use case.