I'm having trouble to connect to a topic on Confluent cloud from a springboot kafka-streams application, the connection failing with a single error line
org.apache.kafka.common.errors.TransactionalIdAuthorizationException: Transactional Id authorization failed.
I believe this is because the topic is configured to allow transactions with a specified transactial-id (https://docs.confluent.io/current/kafka/authorization.html#enabling-authorization-for-idempotent-and-transactional-apis). My service being a streaming application, generates a transactional-id in each StreamThread (create producer method in https://github.com/apache/kafka/blob/2.4.1/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java) which is defined as applicationId + "-" + taskId and this does not match the one defined on topic ACL.
Is this a correct, and if so is there a workaround?
You would need to use wildcard ACLs by only specifying the prefix, ie, <application.id>- for the corresponding ACL, using --resource-pattern-type prefixed
Cf. https://docs.confluent.io/current/streams/developer-guide/security.html