Search code examples
elasticsearchlogginglogstashlogstash-groklogstash-configuration

grok match : parse log file for time only using pattern or match

I want to parse the below mentioned line from log file.

03:34:19,491 INFO [:sm-secondary-17]: DBBackup:106 - The max_allowed_packet value defined in [16M] does not match the value from /etc/mysql/my.cnf [24M]. The value will be used.

After parse, the output must be : 

Time :  03:34:19
LogType : INOF
Message : [:sm-secondary-17]: DBBackup:106 - The max_allowed_packet value defined in  [16M] does not match the value from /etc/mysql/my.cnf [24M].  The value will be used.

Ignore : ,491 (comma and 3 digit number).

Solution

  • Grok filter config should be like this for parsing the mentioned log.

    ...
filter {
    grok {
        match => {"message" => "%{TIME:Time},%{NUMBER:ms} %{WORD:LogType} %{GREEDYDATA:Message}"}
        remove_field => [ "ms" ]
    }
}
...