Search code examples
elasticsearchnestlogstash-grokgrok

Repeat grok pattern


I have a message like this "Something word word 20/07/2018 word word 25/04/2015".How can I use a grok pattern or a customer pattern to take all dates and add them into a new field which is an array?

I tried with a custom regex pattern but when I specify the global flag, ES doesn't recognize "/g" flag.The problem is that I don't know how many dates I will have in a document because are invoices or other type of docuemtns which have a lot of dates/numbers inside.


Solution

  • Since you have stated in the comment section that you are free in your implementation, I would solve this with logstashs ruby filter and with the help of this article:

    https://zzamboni.org/post/capturing-multiple-matches-in-ruby/

    (Sorry for not formatting the link but I'm on my mobile phone right now.)

    The article describes how you match a string against a regex and store all found values in an array.

    So the filter would look something like this (untested):

    filter{
      ruby{
        code => '
          my_string = event.get("my_field") 
          my_array = my_string.scan(/[0-9]+\/[0-9]+\/[0-9]+/)
          event.set("my_array_field_name", my_array)
        '
      }
    }
    

    By playing around with this skeleton you should be able to solve the issue. Also take a look at the documentations.