logstash multiline pattern to detect lines that should be aggregated

Question

I would like to detect lines that does not contain date and time in the following format which should be aggregated to the previous line:

28.04.2017;15:13:30.276

So I created a custom grok pattern like that:

MLINE %{MONTHDAY}\.%{MONTH}\.%{YEAR};%{TIME}

And this is my multiline configuration:

        codec => multiline {
            pattern => "%{MLINE}"
            negate => true
            what => "previous"
        }

What I expected is that if the log line coming to logstash does not contain %{MLINE}, then it should be aggregated to the previous line.

But this is not happening and I do not understand why. Basically every log line is considered to be aggregated, and logstash will soon end due to cache limit.

What I am doing wrong?

Answer 1

The %{MONTH} pattern matches words such as Aug, Feb, September. You want the %{MONTHNUM} pattern.

The pattern you want is:

MLINE %{MONTHDAY}\.%{MONTHNUM}\.%{YEAR};%{TIME}

I tested this using the grok constructor which has a multiline tool.