Multiple Kafka Streams with different topics ApplicationId

Question

Given a single component which can have multiple instances, and the following structure:

1. Stream1[Topic1, destination1]
2. Stream2[Topic2, destination2]

where destination is a Queue and all the links will be 1:1.

Do we need to set the same applicationId for each KafkaStream? It is known that applicationId will generate client.id and group.id which are important for how the partitions are assigned. Couldn't find anything in the official documentation.

Answer 1

If your program is different (ie, different Topology), you need to use different application.id config. Using the same application.id config requires that all instances execute the exact same Topology.

Of course, you could also build a single Topology that processed both topics at once:

StreamsBuilder builder = ...

builder.stream("topic1")...to("destination1");
builder.stream("topic2")...to("destination2");