I'm stuck on a similar problem seen on this post, but can't find a solution : https://github.com/elastic/curator/issues/1513
To snapshot my Elasticsearch cluster (7.7.1), I use curator (5.8) to daily snapshot all indices. I realised today that only my indices starting with "." are being snapshoted by Curator.
If I use the curator-cli, all indices are indeed seen by curator and snapshoted.
I tried to remove all filters in my action file, replaced them by :
filters:
- filtertype: none
Nothing seems to work, my dry-runs always end up listing all indices beggining with a dot.
This is my action file :
---
actions:
1:
action: snapshot
description: >-
Snapshot all indices
options:
repository: backup
name: testbackup6
ignore_unavailable: False
include_global_state: True
partial: False
wait_for_completion: True
skip_repo_fs_check: False
disable_action: False
filters:
- filtertype: none
Curator logs (I have anonymized some results)
2021-01-08 18:34:44,021 INFO DRY-RUN: snapshot: testbackup6 in repository backup with arguments: {'ignore_unavailable': False, 'include_global_state': True, 'partial': False, 'indices': '.apm-XXX,.apm-customXXX,.async-sXXX,.kibana_1,.kibana_task_manager_1,.monitoring-alerts-7,.monitoring-es-7-2021.01.02,.monitoring-es-7-2021.01.03,.monitoring-es-7-2021.01.04
...
,.triggered_watches,.watches'}
I went to see the DEBUG logs, and the indices lifecycle seems to be a problem. Here are some accepted/rejected indices :
2021-01-08 19:54:07,925 DEBUG curator.indexlist __not_actionable:39 Index XXXX_supervision-server_logs-2020.12.31-000014 is not actionable, removing from list.
2021-01-08 19:54:07,925 DEBUG curator.indexlist __excludify:58 **Removed** from actionable list: XXX_supervision-server_logs-2020.12.31-000014 has index.lifecycle.name XXX_supervision-server_logs-policy
2021-01-08 19:54:07,925 DEBUG curator.indexlist __actionable:35 Index .monitoring-es-7-2021.01.05 is actionable and remains in the list.
2021-01-08 19:54:07,925 DEBUG curator.indexlist __excludify:58 **Remains** in actionable list: index.lifecycle.name is not set for index .monitoring-es-7-2021.01.05
2021-01-08 19:54:07,925 DEBUG curator.indexlist __not_actionable:39 Index XXX_logs-2021.01.05-000019 is not actionable, removing from list.
Has anyone experienced this ? I can't see the link between indices having ILM policies and curator not matching them.
I can't find a workaround with regex to help me match all my indices. With the same "filtertype: none" on curator-cli, everything is OK.
Thanks a lot
I just found it ><
"allow_ilm_indices: True" must be added in the action file in order to show all indices...
The curator_cli has this option on True by default, which is not the case of curator itself.