we are using a custom IIS wc3 log, so have changed grok patterns for fit it in the file :
\filebeatXXXX\module\iis\access\ingest\pipeline.yml :
- grok:
field: message
patterns:
- '%{TIMESTAMP_ISO8601:iis.access.time} (?:-|%{IPORHOST:destination.address}) (?:-|%{WORD:http.request.method}) (?:-|%{NOTSPACE:url.path}) (?:-|%{NUMBER:destination.port:long}) (?:-|%{NOTSPACE:user.name}) (?:-|%{IPORHOST:source.address}) (?:-|HTTP/%{NUMBER:http.version}) (?:-|%{NUMBER:http.response.status_code:long}) (?:-|%{NUMBER:iis.access.sub_status:long}) (?:-|%{NUMBER:iis.access.win32_status:long}) (?:-|%{NUMBER:http.response.body.bytes:long}) (?:-|%{NUMBER:http.request.body.bytes:long}) (?:-|%{NUMBER:temp.duration:long})?'
ignore_missing: true
The pattern work in the elastic Dev Tools using Grok Debugger. this is the sample data that I have take from the iis logs.
2021-07-14 06:51:45 19.16.2.8 POST /api/DocumentMainReport/GetReport 443 demouser 19.18.2.1 HTTP/2 200 0 0 1093 1398 96
But there is a still an error "Provided Grok expressions do not match field value...." in elastic discover logs.
What am I missing ?
After you update the ingest\pipeline.yml you need to send it to the server for update using :
./filebeat setup --pipelines -modules="{{ your module name }}"