How to make Spring kafka client distributed

Question

I have messages coming in from Kafka. So I am planning to write a listener and "onMessage". I want to process it and push it in to solr.

So my question is more architectural, like I have worked on web apps all my career, so in big data how to deploy the spring kafka listener, so I can process thousands of messages a second.

1. How do I make my spring code use multiple nodes to distribute the load?
2. I am planning to write a SpringBoot application to run in a tomcat container.

Answer 1

If you use the same group id for all instances, different partitions will be assigned to different consumers (instances of your application).

So, be sure that you specified enough partitions in the topic you are going to consume.