I'm using a FIX filter plugin to process some of our FIX logs. In those messages we receive multiple custom fields. This is outside of the grok filter. I pass the message I care about into this secondary fix plugin
Some of our messages for example look like this:
"unknown_fields" => [
[0] "5000",
[1] "9723",
],
"5000" => "FOOBARVAL",
"9723" => "BAZBOOHUM",
"IDSource" => "RIC_CODE",
Is there a way that I can remove tags with mutate
or some other filter based on a regular expression(^\d+$
)?
More specifically, is there a way that I can remove all of the integer fields that I know will be custom FIX fields (eg. 5000)?
I appreciate the other answer, but I ended up using the prune
filter plugin.
prune {
blacklist_names => ["[0-9]+", "unknown_fields", "tags"]
}