Search code examples
jsonlogstashgroklogstash-grok

logstash grok remove fqdn from hostname and igone ip


my logstash input receive jsons that look like that:

{"src":"comp1.google.com","dst":"comp2.yehoo.com","next_hope":"router4.ccc.com"}

and also the json can look like this ( some keys can hold ip instead of host name:

{"src":"comp1.google.com","dst":"192.168.1.20","next_hope":"router4.ccc.com"}

i want to remove the fqdn and if its contain ip (ignore it)to leave it with the ip

i tried this but its not working

filter {
   grok  {
           match => {
               "src" => "%{IP:src}"
               "src" => "%{WORD:src}"
           }
           overwrite => ["src"]
           break_on_match => true
         }
    grok  {
           match => {
               "dst" => "%{IP:dst}"
               "dst" => "%{WORD:dst}"
           }
           overwrite => ["dst"]
           break_on_match => true
         }
     grok  {
           match => {
               "next_hope" => "%{IP:next_hope}"
               "next_hope" => "%{WORD:next_hope}"
           }
           overwrite => ["next_hope"]
           break_on_match => true
         }
}

this filter working well on the first json. but this not working for the second json ( the dst key) i get this result:

{
   "src" => "comp1",
   "dst" => "192", 
   "next_hope" => "router4"
}

i want dst field will remain with the original value because he has ip address and not a host name.

the result i expect is:

    {
      "src" => "comp1",
       "dst" => "192.168.1.20", 
       "next_hope" => "router4"
     }

any idea? also is there any possibility to do all this trick in 1 grok filter?


Solution

  • Your problem is that the regex for WORD matches a number. The easiest thing to do would be to protect the grok's so that they don't run for IP addresses:

    if [src] !~ /\d+\.\d+\.\d+\.\d+/ {
       grok  {
           match => {
               "src" => "%{WORD:src}"
           }
           overwrite => ["src"]
         }     
    }
    

    And repeat that for the other fields.