Search code examples
logstashlogstash-grok

grok multiple messages and process them with different tags


I want to make a filter in Logstash(version 2.4) with different matches in the same grok. I would like to add different tags depending on the match. Basically, I receive three different message pattern: "##MAGIC##%message" "##REAL##%message" "%message" I am trying to do is:

 grok {
 match => {"message" => "##MAGIC##%{GREEDYDATA:magic_message}"}
 match => {"message" => "##REAL##%{GREEDYDATA:real_message}"}
 match => {"message" => "%{GREEDYDATA:basic_message}"}
 if [magic_message]{
    overwrite => [ "message"]  
    add_tag => ["Magic"]
 } else if [real_message]{
    overwrite => [ "message"]  
    add_tag => ["Real"]
 }else{
   overwrite => [ "message"]  
    add_tag => ["Basic"]
 }

But, I got this compile fails:

    The given configuration is invalid. Reason: Expected one of #, => at line 34, column 9 (byte 900) after filter {
  grok {
     match => {"message" => "##MAGIC##%{GREEDYDATA:magic_message}"}
     match => {"message" => "##REAL##%{GREEDYDATA:real_message}"}
     match => {"message" => "%{GREEDYDATA:basic_message}"}
     if  {:level=>:fatal}

Solution

  • The logstash configuration syntax does not work like this.

    This should work better (under the assumption that you want to replace message by magic_message/real_message):

    grok {
        match => {"message" => [ "##MAGIC##%{GREEDYDATA:magic_message}",
                                 "##REAL##%{GREEDYDATA:real_message}", 
                                 "%{GREEDYDATA:basic_message}"]}
    }
    if [magic_message] {
        mutate {
            replace => { "message" => "%{magic_message}" }
            add_tag => ["Magic"]
        }
    } else if [real_message] {
        mutate {   
            replace => { "message" => "%{real_message}" }
            add_tag => ["Real"] 
        }
    } else {
        mutate {
            add_tag => ["Basic"] 
        }
    }