Exactly-once semantics in spring Kafka

Question

I need to apply transactions in a system that comprises of below components:

1. A Kafka producer, this is some external application which would publish messages on a kafka topic.
2. A Kafka consumer, this is a spring boot application where I have configured the kafka listener and after processing the message, it needs to be saved to a NoSQL database.

I have gone through several blogs like this & this, and all of them talks about the transactions in context of streaming application, where the messages would be read-processed-written back to a Kafka topic.

I don't see any clear example or blog around achieving transactionality in the use case similar to mine i.e. producing-processing-writing to a DB in a single atomic transaction. I believe it to be very common scenario & there must be some support for it as well.

Can someone please guide me on how to achieve this? Any relevant code snippet would be greatly appreciated.

Answer 1

in a single atomic transaction.

There is no way to do it; Kafka doesn't support XA transactions (nor do most NoSQL DBs). You can use Spring's transaction synchronization for best-effort 1PC.

See the documentation.

Spring for Apache Kafka implements normal Spring transaction synchronization.

It provides "best efforts 1PC" - see Distributed transactions in Spring, with and without XA for more understanding and the limitations.