Search code examples
javaapache-kafkaspring-kafka

how does kafka batch listener concurrency work with multiple topics


Let's say I have a batch listener that has a concurrency of 30 and is listening to a single topic "topic1". like so

    @Bean
    public KafkaListenerContainerFactory<ConcurrentMessageListenerContainer<String, String>> batchKafkaListenerContainerFactory(ConsumerFactory kafkaConsumerFactory) {
        factory.setConcurrency(30);
        return factory;
    }
 @KafkaListener(topics = {"topic1"}, containerFactory = "batchKafkaListenerContainerFactory")

Now I want to add another topic to be listened to like so

 @KafkaListener(topics = {"topic1","topic2"}, containerFactory = "batchKafkaListenerContainerFactory")

does that mean there's only 15 threads available for each topic now? (i.e. threads split amongst the topics) Or is it still 30 threads for each topic? And does the addition of "topic2" affect the performance of "topic1"?


Solution

  • Partitions from both topics will be distributed across all 30 consumers (as long as there are at least 30 partitions in each topic).

    See the partitions assigned logs.