I have a use case where I need to send 3 messages to 3 different topics within a single transaction. The issue is that one of the topics is compacted, and I'm pretty new to Kafka transactions so I'm not really sure how transaction cancelling works.
My question is: what actually happens if the transaction fails or aborts (application crash, exception etc)? Will the records of the aborted transaction eventually be removed from the compacted topic's log (like null records)? Is it same with non-compacted topics?
Thanks.
It's a bit late but according to my experience Kafka keeps messages from aborted transactions in the log but marks them with a special marker so that consumer using read_committed isolation level would filter them out but consumer that uses read_uncommitted would still see them.
This could lead to interesting issues like this.
Based on the fact that aborted messages are filtered at the consumer level I think that they should fall under general compaction logic and would be compacted if there's a message with the same key further down the topic but aborted message shouldn't override last committed message with the same key because that would violate guarantees Kafka provides for transactions.