Search code examples
apache-kafkaspring-kafkaspring-kafka-testkafka-transactions-api

transaction-id-prefix need to be identical for same application instance in spring kafka 3?

I understand in spring kafka 3, I need a unique transaction-id-prefix for applications running with multiple instances. here is my configuration:

fun getTxnIdPrefix() {
    return "$hostname-mytransaction-id-prefix-${UUID.randomUUID()}"
}

in this case:

hostA-mytransaction-id-prefix-abcd123xxx
app crash and restart
hostA-mytransaction-id-prefix-mnbmgjh698

does transaction-id-prefix need to be identical for same instance in case application crash/restart?

I saw from confluent blog transactions-apache-kafka suggests the transaction-id-prefix need to be identical before/after application instance crash/restart, in order to look for any pending transaction?

Solution

  • It can be different; however, there will be a delay if there was an in-progress transaction when the instance went down. That transaction will have to time out before a consumer will receive later records.

    See https://kafka.apache.org/documentation/#producerconfigs_transaction.timeout.ms