Search code examples
regexlogstash-grokrouter-os

Regular expression to get topics from MikroTik logs


I have logs which looks like this:

system,info,account user admin logged out from 192.168.1.9 via local
system,info log rule added by admin

Every line begins with comma-separated list of topics and after first space list ends. There can be one, two, three or more topics in list. I need to get topics as group of values, like it is [ "system", "info", "account" ] for first line and [ "system", "info" ] for second.

I was trying to extract list first with use ^\S+ and then [^,]+ on first regex result. It works OK but maybe there is the way which allows to do that with one regex?

I want one-line regex because I'm going to use that regex in Grok pattern to add these topics as tags. Grok uses Oniguruma regex engine.


Solution

  • The solution was to use mutate { split ... merge ... } after groking topics part from the message.

    filter {
        grok {
            patterns_dir => [ "/etc/logstash/patterns" ]
            match => { "message" => "(?<mttopics>^\S+) %{GREEDYDATA:message}" }
            overwrite => [ "message" ]
        }
        mutate {
            split => { "mttopics" => "," }
            merge => { "tags" => "mttopics" }
            remove_field => [ "mttopics" ]
        }
    }