Search code examples
elasticsearchnestelasticsearch-curator

Elasticsearch curator delete action - delete "n" older indices after rollover?


The following elasticsearch curator (below) set-up in curator-actions.yml is configured to delete indices based on age filter but I would like to set-up a config that works with rollover as follows:

Only keep the last index and new created rolled-over index -> thus deleting all other indices after a rollover is successful. What is the best way to accomplish this?? Possible with NEST in code?

Here is my current delete action...any help is much appreciated, thanks!

in curator-actions.yml

action: delete_indices
description: >-
  Delete indices older than 3 days (based on index creation date)
options:
  ignore_empty_list: True
  continue_if_exception: True
filters:
- filtertype: pattern
  kind: prefix
  value: applogging-test
- filtertype: age
  source: creation_date
  direction: older
  unit: days
  unit_count: 3

Solution

  • It's actually easier than you think. You can keep the two most recent indices quite easily with the count filter. The following example incorporates both the rollover action and the delete_indices action immediately after it (I used the conditions you supplied in the comment above--tune your rollover conditions appropriately):

    actions:
      1:
        action: rollover
        description: Rollover index associated with alias name
        options:
          name: aliasname
          conditions:
            max_age: 7d
            max_docs: 1000
            max_size: 5gb
      2:
        action: delete_indices
        description: Keep only the two most recent indices
        options:
          ignore_empty_list: true
        filters:
        - filtertype: pattern
          kind: prefix
          value: applogging-test
        - filtertype: count
          count: 2
    

    Now, this presupposes that all indices matching prefix applogging-test will be rollover-style, and will increment numerically. You can add other options or filters as needed, though.