Search code examples
elasticsearchlogstashlogstash-grok

How to parse json in logstash /grok from ansible_results


I have below messages from ansible_results which i'm trying to parse, what basically i need is to cut the down the field after "msg": from the below messages.

Log sample:

2019-05-07 07:56:06,374 p=7743 u=root |  fatal: [xxxxx]: FAILED! => {"changed": false, "msg": "The system may not be mirrored  according to the xxxx default mirror policy."}
2019-05-07 07:56:06,402 python-logstash-logger TASK FAILED | fail | HOST | xxxxxxx | RESULT | {"changed": false, "msg": "The system may not be mirrored  according to the xxx default mirror policy."}

I'm trying follows but not getting idea to accomplish this:

%{TIMESTAMP_ISO8601:time} p=%{INT:process} u=%{USER:user}|%{SPACE}falal:%{SPACE}%{WORD:fatal}%{SPACE}%{UNIXPATH: FAILED*?}

Desired:

Segregated msg and last message body into two different fields..

msg    The system may not be mirrored  according to the xxxx default mirror policy.

Any expertise help will be much appreciated.


Solution

  • Since you've got two very different log type, I used two different grok pattern:

    grok{
     match => [
       "%{TIMESTAMP_ISO8601:time}.*p=%{INT:process} u=%{USER:user}.*%{WORD:result}! =>.*"msg": "%{GREEDYDATA:msg}"\}$",
       "%{TIMESTAMP_ISO8601:time}.*\|.*\|%{SPACE}%{GREEDYDATA:Host}%{SPACE}\|.*\|.*\|.*"msg": "%{GREEDYDATA:msg}"\}$
     ]
    }
    

    The first pattern with the first log line:

    process     7743
    result  FAILED
    msg     The·system·may·not·be·mirrored··according·to·the·xxxx·default·mirror·policy.
    time    2019-05-07·07:56:06,374
    user    root 
    

    The second pattern with the second log line:

    time    2019-05-07·07:56:06,402
    Host    HOST·
    msg     The·system·may·not·be·mirrored··according·to·the·xxx·default·mirror·policy.