Search code examples
regexelasticsearchelasticsearch-curator

How to use greedy regexes on Curator filter?


I have set up Curator to delete old Elasticsearch indexes via this filter:

(...)
filters:
- filtertype: pattern
  kind: regex
  value: '^xyz-us-(prod|preprod)-(.*)-'
  exclude:
- filtertype: age
  source: name
  direction: older
  timestring: '%Y.%m.%d'
  unit: days
  unit_count: 7
  exclude:
(...)

However, I realized that Curator uses non-greedy regexes, because this filter catches the index xyz-us-prod-foo-2018.10.11 but not xyz-us-prod-foo-bar-2018.10.11.

How can I modify the filter to catch both indexes?


Solution

  • The answer I gave at https://discuss.elastic.co/t/use-greedy-regexes-in-curator-filter/154200 is still good, though you somehow weren't able to get the results I posted there. Anchoring the end and specifying the date regex worked for me: '^xyz-us-(prod|preprod)-.*-\d{4}\.\d{2}\.\d{2}$'

    I created these indices:

    PUT xyz-us-prod-foo-2018.10.11
    PUT xyz-us-prod-foo-bar-2018.10.11
    PUT xyz-us-preprod-foo-2018.10.12
    PUT xyz-us-preprod-foo-bar-2018.10.12
    

    And ran with this config:

    ---
    actions:
      1:
        action: delete_indices
        filters:
        - filtertype: pattern
          kind: regex
          value: '^xyz-us-(prod|preprod)-.*-\d{4}\.\d{2}\.\d{2}$'
          exclude:
        - filtertype: age
          source: name
          direction: older
          timestring: '%Y.%m.%d'
          unit: days
          unit_count: 7
    

    The results are fully matched:

    2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-preprod-foo-2018.10.12 with arguments: {}
    2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-preprod-foo-bar-2018.10.12 with arguments: {}
    2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-prod-foo-2018.10.11 with arguments: {}
    2018-10-29 20:08:28,120 INFO               curator.utils           show_dry_run:928  DRY-RUN: delete_indices: xyz-us-prod-foo-bar-2018.10.11 with arguments: {}