Filter message based on String

Question

I have below logs in same log file
2019-11-23T14:38:43.495 backendorg [http-nio-8080-exec-45] INFO http-nio-8080-exec-45 SessionController http://localhost:8080/ABC/session/login [email protected] backendorg

2019-11-23T14:38:44.235 backendorg [http-nio-8080-exec-45] INFO http-nio-8080-exec-45 SessionController userSession: backendorg 16CFAFCCFB14D9A3 16E978545E17BFEC 16E978545E1452FF

using below filter to parse the messages above based on String "userSession".

 input {
      file {
            tags => ["stacktrace"]
            type => "error_logs"
            path => ["/Users/znrind-a0053/Downloads/logs/zapp-audit.log"]
            start_position => "beginning"
            sincedb_path => "/tmp/sincedb_file"
            codec => multiline {
            pattern => "^%{TIMESTAMP_ISO8601} "
            negate => true
            what => previous
            }
      }

}
filter {

      if "userSession" in [message]{
        grok {
        match => [ "message",
                 "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{URI:url}%{SPACE}(?<email>[\w.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:[.](?:[0-9A-Za-z][0-9A-Za-z‌​-]{0,62}))*)%{SPACE}%{USERNAME:orgnisation}"]
          }
      } else {

      grok {
      match => [ "message",
               "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVACLASS:javaClass} %{USERNAME:logmessage}:?%{SPACE}%{USERNAME:orgnisation}%{SPACE}%{USERNAME:loginUserId}%{SPACE}%{USERNAME:sessionId}%{SPACE}%{USERNAME:txnId}"]
        }
      }
}
output {
        elasticsearch {
            hosts => "localhost"
            index => "logs"
        }
        stdout{codec => json}
}

But receiving GROK parser error. Any suggestion highly appreciated.

Answer 1

Try this in filter:

filter {

      if "userSession" in [message]{
        grok {
        match => [ "message",
                 "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{USERNAME:logmessage}:?%{SPACE}%{USERNAME:orgnisation}%{SPACE}%{USERNAME:loginUserId}%{SPACE}%{USERNAME:sessionId}%{SPACE}%{USERNAME:txnId}"]
          }
      } else {

      grok {
      match => [ "message",
               "%{TIMESTAMP_ISO8601:timestamp_match} %{USERNAME:orgId} (\[%{DATA:thread}\])?( )?%{LOGLEVEL:level}%{SPACE}%{USERNAME:zhost} %{JAVAFILE:javaClass} %{URI:url}%{SPACE}(?<email>[\w.+=:-]+@[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:[.](?:[0-9A-Za-z][0-9A-Za-z‌​-]{0,62}))*)%{SPACE}%{USERNAME:orgnisation}"]
        }
      }
}