I have tab separated string and I want to extract each fields using grok plugin. The tab separated string is like
http://www.allaboutpc.co.kr 2016110913 d6123c6caa12f08852c82b876bdd3ceceb166d5e 0 0 1 0 /Event/QuizChoice.asp?IdxEvent=3141
I would like to get each fields as url, datetime, hashvalue, count1, count2, count3, count4, path.
I used %{DATA:hashvalue} for 3rd field to extract hashvalue but logstash didn't print hashvalue
Here is my conf file
input {
stdin { }
file {
path => "/Users/Projects/webmastermrinput/20161021/17/*"
codec => plain
}
}
filter {
# tab to space
mutate {
gsub => ["message", "\t", " "]
}
grok {
match => {
'message' => "%{DATA:url} %{NUMBER:datetime2} %{DATA:hashvalue} % {NUMBER:count1} %{NUMBER:count2} %{NUMBER:count3} %{NUMBER:count4} % {URIPATHPARAM:path}'
}
}
}
output {
stdout { codec => rubydebug }
}
Logstash output for input : "http://www.allaboutpc.co.kr 2016110913 d6123c6caa12f08852c82b876bdd3ceceb166d5e 0 0 1 0 /Event/QuizChoice.asp?IdxEvent=3141"
{
"@timestamp" => 2016-11-11T02:26:01.828Z,
"@version" => "1",
"host" => "MacBook-Air-10.local",
"datetime" => "2016110913",
"message" => "http://www.allaboutpc.co.kr 2016110913 d6123c6caa12f08852c82b876bdd3ceceb166d5e 0 0 1 0 /Event/QuizChoice.asp?IdxEvent=3141",
"url" => "http://www.allaboutpc.co.kr"
}
Your grok works perfectly well, you just need to remove the spaces between %
and {
in % {NUMBER:count1}
and % {URIPATHPARAM:path}
'message' => "%{DATA:url} %{NUMBER:datetime2} %{DATA:hashvalue} % {NUMBER:count1} %{NUMBER:count2} %{NUMBER:count3} %{NUMBER:count4} % {URIPATHPARAM:path}'
^ ^
| |
here and here