AWS ElasticSearch: How to apply a policy to an index

Question

We have an AWS ElasticSearch domain and are writing records/documents to it. I've now created an Index State/Lifecycle Management (ISM/ILM) policy in Kibana and I can apply the policy to an Index from within Kibana. I now want to apply that policy when the index is created from within our Java code that handles writes to the Index (using the High Level REST API).

I have found no methods in the High Level REST API that specifically allow assigning a policy to an index, however I think that I should be able to do it using the RequestOptions object that is used when the Index is created. The documentation is pretty thin, but it seems that I should be able to basically insert a key/value into the Index properties. For example, when I inspect the index where I have manually assigned the policy, find the following keys where the policy is assigned.

"settings" : {
  "index" : {
    "opendistro" : {
      "index_state_management" : {
        "policy_id" : "DefaultLifeCyclePolicy_30DayWarm_180DayDelete"
      }
    },

It seems reasonable to assume that i can just insert a similar key into the Index object. The following code seems like it should work. It does run without error, but the RequestOptions does nothing useful.

        boolean isExisting = mAwsClient.indices().exists(new GetIndexRequest(indexNameFull), RequestOptions.DEFAULT);
        if (!isExisting) {
            RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
            builder.addHeader("settings.opendistro.index_state_management.policy_id", mIndexStateMgmtPolicy);
            RequestOptions requestOptions = builder.build();

            CreateIndexRequest request = new CreateIndexRequest(indexNameFull);
            request.mapping(mapping, XContentType.JSON);
            CreateIndexResponse createIndexResponse = mAwsClient.indices().create(request, requestOptions);
        }

So, how can I assign am ISM/ILM policy to an index using Java?

BTW, I have looked at creating an Index Template, which should assign the policy automatically on index creation, but the OpenDistro Kibana does not seem to have that functionality.

Answer 1

BTW, I have looked at creating an Index Template, which should assign the policy automatically on index creation, but the OpenDistro Kibana does not seem to have that functionality.

You can use the index template in the following manner to apply ISM policy at index creation:

PUT _template/template_1
{
  "index_patterns": [
    "test-index*"
  ],
  "settings": {
    "index": {
      "opendistro": {
        "index_state_management": {
          "policy_id": "DefaultLifeCyclePolicy_30DayWarm_180DayDelete"
        }
      }
    }
  }
}

For rollover indices:

1. Make sure you have alias:

POST /_aliases
{
    "actions" : [
        { "add" : { "index" : "test-index-000001", "alias" : "test-index" } }
    ]
} 

2. Template:

PUT _template/template_1
{
  "index_patterns": [
    "test-index*"
  ],
  "settings": {
    "index": {
      "opendistro": {
        "index_state_management": {
          "policy_id": "DefaultLifeCyclePolicy_30DayWarm_180DayDelete",
          "rollover_alias": "test-index"
        }
      }
    }
  }
}