Error in logstash-plain.log cannot create pipeline

Question

Need a little bit of help figuring out why the pipeline is not starting.

[ERROR][logstash.agent] Cannot create pipeline {:reason=>"Expected one of #, => at line 39, column 52 (byte 563) after filter {\n grok {\n patterns_dir =>  \"/etc/logstash/patterns.d\" \n match => {\n \"%{SYSLOGBASE} %{POSTFIXSMTPDCONNECTS}\""}

Here is my grok file:

input {
        file {
        type => "postfix"
        path => "/var/log/maillog"
    }
}

filter {
    grok {
        patterns_dir =>  ["/etc/logstash/patterns.d"]
        match => {
            "%{SYSLOGBASE} %{POSTFIXSMTPDCONNECTS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDACTIONS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDTIMEOUTS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDLOGIN}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDCLIENT}",
            "%{SYSLOGBASE} %{POSTFIXSMTPRELAY}",
            "%{SYSLOGBASE} %{POSTFIXSMTPCONNECT}",
            "%{SYSLOGBASE} %{POSTFIXSMTP4XX}",
            "%{SYSLOGBASE} %{POSTFIXSMTP5XX}",
            "%{SYSLOGBASE} %{POSTFIXSMTPREFUSAL}",
            "%{SYSLOGBASE} %{POSTFIXSMTPLOSTCONNECTION}",
            "%{SYSLOGBASE} %{POSTFIXSMTPTIMEOUT}",
            "%{SYSLOGBASE} %{POSTFIXBOUNCE}",
            "%{SYSLOGBASE} %{POSTFIXQMGR}",
            "%{SYSLOGBASE} %{POSTFIXCLEANUP}"
        }
        named_captures_only => true
    }
}

I'm on logstash v5.6.5. I read that the all the files in the conf.d folder combine when its read. Is there a specific file that I need to look for in order to look at specific line 39 of code.

Answer 1

There is an error on how you wrote your grok filter. Since there are multiple patterns, you should use an array. And you've forgotten to indicate on which field you are doing the match. See the documentation for more information.

The correct way to write your grok filter would be (supposing your want to apply the grok filter on the message field):

grok {
    patterns_dir =>  ["/etc/logstash/patterns.d"]
    match => { 
        "message" => [
            "%{SYSLOGBASE} %{POSTFIXSMTPDCONNECTS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDACTIONS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDTIMEOUTS}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDLOGIN}",
            "%{SYSLOGBASE} %{POSTFIXSMTPDCLIENT}",
            "%{SYSLOGBASE} %{POSTFIXSMTPRELAY}",
            "%{SYSLOGBASE} %{POSTFIXSMTPCONNECT}",
            "%{SYSLOGBASE} %{POSTFIXSMTP4XX}",
            "%{SYSLOGBASE} %{POSTFIXSMTP5XX}",
            "%{SYSLOGBASE} %{POSTFIXSMTPREFUSAL}",
            "%{SYSLOGBASE} %{POSTFIXSMTPLOSTCONNECTION}",
            "%{SYSLOGBASE} %{POSTFIXSMTPTIMEOUT}",
            "%{SYSLOGBASE} %{POSTFIXBOUNCE}",
            "%{SYSLOGBASE} %{POSTFIXQMGR}",
            "%{SYSLOGBASE} %{POSTFIXCLEANUP}"
        ]
    }
    named_captures_only => true
}