Is there a way to stop curator deleting the last index when deleting by time?
actions:
1:
action: delete_indices
description: Delete kube- indices older than 14 days. Ignore the error if there are none and exit cleanly.
options:
disable_action: False
ignore_empty_list: True
filters:
- filtertype: pattern
kind: prefix
value: kube-
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 14
I have this which works great to keep a currently running K8s cluster logs in check. However when we move AWS region the log name changes e.g. from kube-eu-west-1-<date> to kube-eu-west-2-<date>.
Curator diligently cleans up all the data after 14 days. What I'd like is to prevent it from removing the last entry for a particular index, so there is always a record of what happened the last time the cluster was in that region.
(It would also "fix" some less well written pieces of code that throw errors when the data they expect to be there has legitimately gone away).
You could use the count filter:
filters:
# your existing filters go BEFORE the count filter...
- filtertype: count
count: 1
This example should exclude (in spite of the exclude: false) the most recent index from the list of actionable indices, preserving it. If this is not the index you want to exclude, explore using exclude: true/false (default is true), and/or reverse: true/false (default is true) until it excludes the index you want.
NOTE: Always use the --dry-run flag to test your filters before deploying them on actual data. Iterate until it looks right. Use of loglevel: DEBUG in your client settings will show how filters make their decisions, if that helps.