Search code examples
logstashelastic-stacklogstash-grok

Using separators between the grok expressions

I have to write a grok expression for the given line : 

2019-11-14 17:29:20 fqm.kfa::(1087651)

I am able to write grok for the part of line before :: as follows : 

%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND})?\s*%{WORD:word1}.%{WORD:word2}

and it works perfectly fine. However when I try to combine the part of the line after ::, it shows No matches 

%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND})?\s*%{WORD:word1}.%{WORD:word2}::(%{INT:num})
Solution

  • First, I would suggest instead of breaking down the date into many fields like day, hour, minute, second, etc. (which usually you don't need them separately) to use the TIMESTAMP_ISO8601 data type.

    The second part is problematic not because of the "::" but because of the ( and ). these special characters used by regex. In order to use them you will have to escape them like this: \( and \).

    Like this:

    ^%{TIMESTAMP_ISO8601:event_timestamp}%{SPACE}%{WORD:word1}.%{WORD:word2}::\(%{NUMBER:num:int}\)$

    Notice I have also change the num data type to NUMBER:num:int that will cause ES to treat this data as integer enabling you later to use filters like bigger/smaller than.