Search code examples
elasticsearchlogstashlogstash-grok

Logstash date format grok pattern

I have a log file which has a date time in 'yyyyMMdd_HHmmss_SSS' format. I am successful in parsing this with _ as delimiter and getting as 3 different text field in ES. But I need this to be converted as ISO_8601 so I can query and visualize the data by date or by hour or by minute.

Solution

  • If you don't specifically need ISO-8601, but care more about the events getting a queryable timestamp, the date filter sounds like a better fit for you.

    filter {
  date {
    match => [ "logdate", "yyyyMMdd_HHmmss_SSS" ]
  }
}

    This will set the @timestamp field to be a date-searchable field.

    However, if you really do need Grok to do the work, you'll probably be best suited through using custom regexes.

    (?<logyear>\d{4,})(?<logmonth>\d\d)(?<logday>\d\d)_(and so on)

    This leverages single-digit captures to build your string.