Search code examples
logginglogstashrollover

ILM Policy is rolling over later than configured


I have created the following ILM Policy for testing:

{
  "test-1-m": {
    "version": 6,
    "modified_date": "2022-12-07T11:27:55.648Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "10mb",
              "max_age": "1m"
            }
          }
        },
        "delete": {
          "min_age": "1m",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by": {
      "indices": [
        "monitoring-logstash-alias-000008",
        "monitoring-logstash-alias-000009",
        "monitoring-logstash-alias-000010"
      ],
      "data_streams": [],
      "composable_templates": [
        "monitoring-logstash-alias"
      ]
    }
  }
}

The shard is rolled over but after 10 minutes instead of 1 as configured. The 10 MB threshold is not applied at all.

Am I missing something?


Solution

  • Check your cluster's indices.lifecycle.poll_interval setting, it affect the overall rollover policy of your cluster

    Use the following command in dev tools to determine your cluster's current value:

    GET _cluster/settings
    

    This command will allow you to change the current value to your desired value:

    PUT _cluster/settings
    {
        "transient": {
             "indices.lifecycle.poll_interval": "5s"
        }
    }