Search code examples
filterlogstashgrok

Grok with Logstash - Logs from windows and linux - how?

My Grok filter for LogStash:

bin/logstash -e '
input { stdin { } }
filter {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
}
output { stdout { codec => rubydebug } }'

It is perfect for my Linux logins logs:

Mar  9 14:18:20 ServerName sshd[14160]: pam_unix(sshd:session): session opened for user root by (uid=0)
{
             "message" => "Mar  9 14:18:20 ServerName sshd[14160]: pam_unix(sshd:session): session opened for user root by (uid=0)",
            "@version" => "1",
          "@timestamp" => "2015-03-09T15:08:39.189Z",
                "host" => "elasticsearchservername",
    "syslog_timestamp" => "Mar  9 14:18:20",
     "syslog_hostname" => "ServerName",
      "syslog_program" => "sshd",
          "syslog_pid" => "14160",
      "syslog_message" => "pam_unix(sshd:session): session opened for user root by (uid=0)"
}

The problem are the windows logs (doesn't have brackets), so I can't get the syslog_pid:

Mar  3 08:58:57 ServerName2 Security-Auditing: 4624: AUDIT_SUCCESS Se inici.. sesi..n correctamente en una cuenta. Sujeto: Id. de seguridad: 
{
             "message" => "Mar  3 08:58:57 ServerName2 Security-Auditing: 4624: AUDIT_SUCCESS Se inici.. sesi..n correctamente en una cuenta. Sujeto: Id. de seguridad:",
            "@version" => "1",
          "@timestamp" => "2015-03-09T15:22:50.351Z",
                "host" => "elasticsearchservername",
    "syslog_timestamp" => "Mar  3 08:58:57",
     "syslog_hostname" => "ServerName2 ",
      "syslog_program" => "Security-Auditing",
      "syslog_message" => "4624: AUDIT_SUCCESS Se inici.. sesi..n correctamente en una cuenta. Sujeto: Id. de seguridad:"
}

How can I change the grok filter for both logs (windows and linux) and get the two syslog_pid?

Thanks?

Solution

  • you can make the brackets optional by doing something like [\[]* and [\]]*

    %{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}[\[]*%{POSINT:syslog_pid}[\]]*: %{GREEDYDATA:syslog_message}