How Filebeat Kafka Module consume Kafka Topics?

Question

I want to send messages from Kafka Topic to the Elastic Search. When I looked for the solutions, I came across Filebeat Kafka Module. As far as I understood, that's a module consuming the given topics. After that, I examined it on their GitHub repos. But I didn't understand that how it's consume Kafka topics. Can anyone understand Go codes, can explain to me how it works? Here is the related GitHub repo: https://github.com/elastic/beats/tree/master/filebeat/module/kafka

Answer 1

That's not the consumer, this is

If you'd like to understand how it works, it uses Sarama - https://github.com/Shopify/sarama

Filebeat is just one option, but Confluent has partnered with Elasticsearch to maintain the Elasticsearch Kafka Connector, so if you're more familiar with Java, then you could use that (and it scales better than Filebeat)