I am syncing data from kafka to elascticsearch using fluentd. But fluentd takes 60 seconds to sync data to elasticsearch. I want real time data syncing. Is there any configuration parameter which i will have to include.
i have tried
<source>
@type kafka
brokers localhost:9092
topics xxx
</source>
<match xxx>
@type elasticsearch
scheme http
port 9200
<buffer tag>
@type memory
flush_thread_count 4
</buffer>
</match>
we use flush_interval parameter, like this:
<buffer>
flush_interval 5s
flush_thread_count 4
</buffer>