Search code examples
elasticsearchlogstashkibanaelastic-stacklogstash-grok

Create a new index in elasticsearch for each log file by date

Currently
I have completed the above task by using one log file and passes data with logstash to one index in elasticsearch :

yellow open logstash-2016.10.19 5 1 1000807 0 364.8mb 364.8mb

What I actually want to do

If i have the following logs files which are named according to Year,Month and Date

MyLog-2016-10-16.log
MyLog-2016-10-17.log
MyLog-2016-10-18.log
MyLog-2016-11-05.log
MyLog-2016-11-02.log
MyLog-2016-11-03.log

I would like to tell logstash to read by Year,Month and Date and create the following indexes :

yellow open MyLog-2016-10-16.log
yellow open MyLog-2016-10-17.log
yellow open MyLog-2016-10-18.log
yellow open MyLog-2016-11-05.log
yellow open MyLog-2016-11-02.log
yellow open MyLog-2016-11-03.log

Please could I have some guidance as to how do i need to go about doing this ?

Thanks You

Solution

  • It is also simple as that :

     output {
   elasticsearch {
      hosts => ["localhost:9200"]
      index => "MyLog-%{+YYYY-MM-DD}.log"
   }
 }