Search code examples
logstashlogstash-grok

Logstash grok doesn't match for specific field


I have tab separated string and I want to extract each fields using grok plugin. The tab separated string is like

http://www.allaboutpc.co.kr 2016110913 d6123c6caa12f08852c82b876bdd3ceceb166d5e 0 0 1 0 /Event/QuizChoice.asp?IdxEvent=3141

I would like to get each fields as url, datetime, hashvalue, count1, count2, count3, count4, path.

I used %{DATA:hashvalue} for 3rd field to extract hashvalue but logstash didn't print hashvalue

Here is my conf file

input {
    stdin { }
    file {
        path => "/Users/Projects/webmastermrinput/20161021/17/*"
        codec => plain
    }
}
filter {
    # tab to space 
    mutate {
       gsub => ["message", "\t", " "]
    }
    grok {
        match => {
            'message' => "%{DATA:url} %{NUMBER:datetime2} %{DATA:hashvalue} %    {NUMBER:count1} %{NUMBER:count2} %{NUMBER:count3} %{NUMBER:count4} %      {URIPATHPARAM:path}'
        }
    }
}
output {
    stdout { codec => rubydebug }
}

Logstash output for input : "http://www.allaboutpc.co.kr 2016110913 d6123c6caa12f08852c82b876bdd3ceceb166d5e 0 0 1 0 /Event/QuizChoice.asp?IdxEvent=3141"

{
    "@timestamp" => 2016-11-11T02:26:01.828Z,
    "@version" => "1",
    "host" => "MacBook-Air-10.local",
    "datetime" => "2016110913",
    "message" => "http://www.allaboutpc.co.kr 2016110913  d6123c6caa12f08852c82b876bdd3ceceb166d5e    0   0   1   0   /Event/QuizChoice.asp?IdxEvent=3141",
    "url" => "http://www.allaboutpc.co.kr"
}

Solution

  • Your grok works perfectly well, you just need to remove the spaces between %and { in % {NUMBER:count1} and % {URIPATHPARAM:path}

    'message' => "%{DATA:url} %{NUMBER:datetime2} %{DATA:hashvalue} %    {NUMBER:count1} %{NUMBER:count2} %{NUMBER:count3} %{NUMBER:count4} %      {URIPATHPARAM:path}'
                                                                      ^                                                                         ^
                                                                      |                                                                         |
                                                                     here                                                                    and here