Search code examples
regexlogstashlogstash-grok

Is it possible to use Logstash to remove_field if it does not match a certain value?

We want to filter a log using Logstash by removing fields if the field does not contain "_log". The remove_field syntax is available, but only works for removing a field if it matches a certain condition. 

filter {
  grok {
    remove_field => [ "log_" ]
  }
}
# This works for removing the log_ field, we want to remove everything that does NOT match log_.

Is it also possible to remove a field if it does not match a certain condition?

We tried using a regex that did just that, but that did not work (is it documented somewhere that you cannot use a regex?). Removing all other fields is also an option, but way more effort. We hope someone can help us fitering all fields that do not contain "log_".

Solution

  • The regexp should work:

    filter {
  if [field] !~ /pattern/ {
    mutate {
      remove_field => [ "field" ]
    }
  }
}