Search code examples
elasticsearchlogstash-configuration

create quarterly index in logstash

hi i am trying to create quarterly index in ES using log-stash , i know how to create index weekly in log-stash here is my piece of configuration - 

> output {
>       elasticsearch {
>           hosts => "localhost"
>           index => "logstash-%{+xxxx.ww}"
>           
>           
>       }
>       stdout{}
>     }

but how can i create quarterly index or how we can have month in any variable so i can calculate the quarter . thanks

Solution

  • Date math currently doesn't support specifying quarters Q and an issue is still open to improve upon this.

    Ideally it would be nice if we could circumvent this shortcoming with something like now-3M/3M but multiples of rounding are not supported either.

    Until the issue is resolved, one solution would be to use monthly indices and when a quarter has gone, reindex the three previous monthly indices into a single quarter index.

    Another solution is to compute the quarter beforehand in a Logstash ruby filter and then use it in the elasticsearch output, like this:

    filter {
   ruby {
      code => "event.set('quarter', Date.today.year + '-' + (Date.today.month / 3.0).ceil)"
   }
}
output {
  elasticsearch {
      hosts => "localhost"
      index => "logstash-%{quarter}"
  }
}