Search code examples
amazon-web-serviceslogstashdevopslogstash-grok

Grok Conditional pattern in Logstash yaml (if..else)


I would like to do grok pattern based on filter conditions using if-else. I've tried but I got the wrong output. Please help me on identifying the error here.

filter {
  if ([service] =~ /service-name/) {
    grok {
      match => { "message" => "\[%{TIMESTAMP_ISO8601:time}\] "}
    }
 }
 else{
   grok {
     match => { "message" => "\[%{TIMESTAMP_ISO8601:time}\] \[%{WORD:logLevel}\] \[\{\"id\":%{DATA:id},\"data\":%{DATA:response}\]"}
     }
   }
 }
}

and below is the log for which I want to do grok pattern

[11/Jul/2022:09:20:09 +0000]|| 00.0.0.000 || "-" || "POST /api/v1/path/subpath HTTP/1.1" || 200 || 630 || 0.005 || "okhttp/4.9.0"

Solution

  • In your first grok the timestamp is HTTPDATE format instead of ISO8601 format, you must change it. Delete also the white space after \]

    Here my suggestions

    filter {
      if ([service] =~ /service-name/) or ([service] =~ /service-name1/) {
        grok {
          match => { "message" => "\[%{HTTPDATE:time}\]\|\| %{IP:ip} \|\| "%{DATA}" \|\| "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))" \|\| %{NUMBER:response} \|\| (?:%{NUMBER:bytes}|-) \|\| (?:%{NUMBER:duration}|-) \|\| %{QS:user-agent}"}
        }
     }
     else{
       grok {
         match => { "message" => "\[%{TIMESTAMP_ISO8601:time}\] \[%{WORD:logLevel}\] \[\{\"id\":%{DATA:id},\"data\":%{DATA:response}\]"}
         }
       }
     }
    }