Search code examples
elasticsearchlogstashkibanaelk

Logstash second translate filter ignored

I have a problem with Logstash translate filter and can't find any solution for that. I want to create a field for vendor names and i take first 6 characters from source and destination mac and compare these values to my dictionary. Filter snippet:

if [source][mac] or [destination][mac] {
    grok {
      tag_on_failure => [ "vendorsmac_grok" ]
      pattern_definitions => {
        SRC_VENDOR => "^(?<[vendor][source][mac]>.{6}).+"
        DST_VENDOR => "^(?<[vendor][destination][mac]>.{6}).+"
      }
      match => {
        "[source][mac]" => "%{SRC_VENDOR}"
        "[destination][mac]" => "%{DST_VENDOR}"
      }
    }
    
    translate {
      dictionary_path => "/etc/logstash/dictionary/mac_vendor.yml"
      field => "[vendor][source][mac]"
      destination => "[vendor][source][name]"
      fallback => "unknown device SRC VENDOR"
      add_tag => "Vendor_SRC_MAC"
    }

    translate {
      dictionary_path => "/etc/logstash/dictionary/mac_vendor.yml"
      field => "[vendor][destination][mac]"
      destination => "[vendor][destination][name]"
      fallback => "unknown device DST VENDOR"
      add_tag => "Vendor_DST_MAC"
    }
  }

When I have these two matches in my grok, translate takes only the second one and this is the result:

"vendor" => {
        "destination" => {
            "name" => "unknown device DST VENDOR",
             "mac" => "9077EE"
        }

But there's no information about Source MAC in my event, it is completely ignored. When I comment Destination MAC:

grok {
      tag_on_failure => [ "vendorsmac_grok" ]
      pattern_definitions => {
        SRC_VENDOR => "^(?<[vendor][source][mac]>.{6}).+"
        #DST_VENDOR => "^(?<[vendor][destination][mac]>.{6}).+"
      }
      match => {
        "[source][mac]" => "%{SRC_VENDOR}"
        #"[destination][mac]" => "%{DST_VENDOR}"
      }
    }

I have my [vendor][source][name] field resolved properly:

"vendor" => {
        "source" => {
            "name" => "Wistron",
             "mac" => "54EE75"
        }

Can someone point me what's wrong here? Am I facing some kind of bug? Tested on Logstash 7.10.2-1 and 7.17.2-1.

Solution

  • Quick look on the documentation... Option break_on_match in grok is set to true by default. I changed it to false and it worked. I've lost a lot of time for that :D