My understanding is I can filter data using stream and put it to specific topics.
Problem : The producer sends data with field country. Then stream processing filters these data and puts to topics by country code.
As result those consumers who are subscribed to specific countries(code) would get message.
Problem is it requires a lot of topics by count of countries. And in the feature I need to do the same with countries.
How to organize it in Kafka and filter data?
You have few options here :
Kafka Streaming : With kafka streaming you can filter data as per your need and write it to the new topics. Consumers can consume messages from those new topics.
Filter Data on the Consumer Side : You consume the data and filter the data as per required criteria on the consumer side.
Use Separate partitions for separate country code : You define total partitions of this topic as per the number of country codes and make country code as key. Now make your consumers direct to right partition for consuming country specific messages.