Logstash 1.4.2 grok filter: _grokparsefailure

Question

1. i am trying to parse this log line: - 2014-04-29 13:04:23,733 [main] INFO (api.batch.ThreadPoolWorker) Command-line options for this run: here's the logstash config file i use:

input {
        stdin {}
}

filter {
 grok {
    match => [ "message", " - %{TIMESTAMP_ISO8601:time} \[%{WORD:main}\] %{LOGLEVEL:loglevel} %{JAVACLASS:class} %{DATA:mydata} "]
  }

    date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  }

output {
  elasticsearch {
    host => "localhost"
  }
  stdout { codec => rubydebug }
}

Here's the output i get:

{
       "message" => " - 2014-04-29 13:04:23,733 [main] INFO (api.batch.ThreadPoolWorker) Commans run:",
      "@version" => "1",
    "@timestamp" => "2015-02-02T10:53:58.282Z",
          "host" => "NAME_001.corp.com",
          "tags" => [
        [0] "_grokparsefailure"
    ]
}

Please if anyone can help me find where the problem is on the gork pattern. I tried to parse that line in http://grokdebug.herokuapp.com/ but it parses only the timestamp, %{WORD} and %{LOGLEVEL} the rest is ignored!

Answer 1

There are two error in your config.

First

The error in GROK is the JAVACLASS, you have to include ( ) in the pattern, For example: \(%{JAVACLASS:class}\.

Second

The date filter match have two value, first is the field you want to parse, so in your example it is time, not timestamp. The second value is the date pattern. You can refer to here

Here is the config

input {
        stdin {

        }
}

filter {
        grok {
                match => [ "message", " - %{TIMESTAMP_ISO8601:time} \[%{WORD:main}\] %{LOGLEVEL:loglevel} \(%{JAVACLASS:class}\) %{GREEDYDATA:mydata}"
                ]
        }
        date {
                match => [ "time" , "YYYY-MM-dd HH:mm:ss,SSS" ]
        }
}

output
{
        stdout {
                codec => rubydebug
        }
}

FYI. Hope this can help you.