I am trying to parse this log format:
http://localhost:8080/,200,OK,11382,date=Mon 27 Apr 2015 12:56:33 GMT;newheader=foo;connection=close;content-type=text/html;charset=ISO-8859-1;server=Apache-Coyote/1.1;
with this config file:
input {
stdin{}
}
filter {
grok {
match => [ "message" , "%{URI:uriaccessed},%{NUMBER:httpcode},%{WORD:httpcodeverb},%{NUMBER:bytes},date=%{TIMESTAMP_ISO8601:logtimestamp};%{GREEDYDATA:msg}"]
}
mutate{
convert => ["httpcode","integer"]
convert => ["bytes","integer"]
}
date {
locale => "en"
match => [ "logtimestamp" , "EEE dd MMM yyy HH:mm:ss" ] #Mon 27 Apr 2015 12:56:33 GMT
}
}
output {
stdout { codec => rubydebug }
}
However, I am getting a grok prase failure, I am not sure what the problem is. cant seem to pin point the pattern that is causing the problem. Any thoughts/comments would be appreciated.
TIMESTAMP_ISO8601 matches:
%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}?
and your date is not in that format. There doesn't seem to be a predefined pattern for you, so here's one that will work:
%{DAY} +%{MONTHDAY} %{MONTH} %{YEAR} +%{TIME} %{WORD}
Note that %{TZ} doesn't like GMT, so I used %{WORD}.
Good luck.