Search code examples
elasticsearchlogstash

logstash put all log files in one Elasticsearch index and create a new index per each log file day for Elasticsearch in Logstash configuration

This is the naming convention of my log files which looks like this:

adminPortal-2021-10-10.0.log
adminPortal-2021-10-27.0.log

I need to publish them to different indices that match the log file date, but Elasticsearch publishes logs from all log files into one index.

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "admin-%{+YYYY-MM-dd}"
  }
}
Solution

  • A sprintf reference to a date, like %{+YYYY-MM-dd} always uses the value of the @timestamp field. If you want it to use the value from the log entry you will need to parse the timestamp out of the [message] field, possibly using grok, and then parse that using a date filter to overwrite the default value of the @timestamp field (which is Time.now).