Search code examples
timestamplogstashlogstash-grok

Using log timestamp as @timestamp in elasticsearch


I am trying to import jetty logs into elasticsearch via filebeat.

the logs has the following format:

Sep 06, 2017 8:06:32 PM com.nphase.redcapcloud.server.managers.rules.RuleRunAsync executeSync
INFO: Rule execution start: 3-6 Month Follow Up - Hide Imaging Form

logstash config looks like this:

filter {
  if "jetty" in [service] {
    grok {
      match => { "message" => ["%{MONTH:[system][jetty][month]} %{MONTHDAY:[system][jetty][day]}, %{YEAR:[system][jetty][year]} %{TIME:[system][jetty][time]}%{CRON_ACTION:[system][jetty][day_period]} %{NOTSPACE:[system][jetty][class]} %{WORD:[system][jetty][method]}\n%{GREEDYMULTILINE:[system][jetty][multiline]}"]}
      pattern_definitions => {
        "GREEDYMULTILINE" => "(.|\r|\n)*"
      }
   }
   mutate {
     add_field => {
       "timestamp_match" => "%{[system][jetty][month]} %{[system][jetty][day]}, %{[system][jetty][year]} %{[system][jetty][time]}%{[system][jetty][day_period]}"
     }
   }
   date {
     match => [ "timestamp_match",
                "MM dd, YYYY KK:mm:ss aa",
                "MM d, YYYY KK:mm:ss aa" ]
     timezone => "UTC"
   }
  }
}

I have tried to match the date with the timestamp_match field but it is not working. Any idea of what can be wrong?


Solution

  • date {
         match => [ "timestamp_match",
                    "MMM dd, YYYY KK:mm:ss aa",
                    "MMM d, YYYY KK:mm:ss aa" ]
        timezone => "UTC"
       }
    

    Date match needed MMM since the month had 3 letters.