Search code examples
elasticsearchlogstashfilebeat

Logstash output from filebeat. What is 'index' configuration option?

This is an excerpt taken from filebeat config for logstash output here

enter image description here

I'm wondering what does index have to do with logstash. In my logstash configuration itslef, if I redirect logs to ElasticSearch I believe my logs will be indexed under "logstash-%{+YYYY.MM.dd}" as the documentation says here.

So why is there an option to set index for filebeat's logstash output?

Solution

  • This option exists because you can use that value in the logstash configuration for the index name as logstash also have an index option to set the index name when sending data to elasticsearch.

    The index option value from filebeat is passed as a metadata field to logstash, and you can configure your elasticsearch output in logstash to use this field as the index name.

    output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}" 
  }
}

    The default value for the index option is the beats name, filebeat, metricbeat, heartbeat and auditbeat for example, but if you set it to logs-prd in your beat configuration file for example, logstash will use that value as the index name.

    If you send your data to logstash before sending it to elasticsearch, it is always logstash that will set the index name using the index option, if you don't set the index option, it will use the default value, which is simple logstash on newer versions.