How to add "cold data node" to elasticsearch cluster using helm?

Question

I would like to add COLD data node (NOT data node) to my elasticsearch cluster using helm:

My values.yaml:

...
roles:
  master: "false"
  ingest: "false"
  data: "false"
  remote_cluster_client: "false"
  ml: "false"
  data_cold: "true"
...

but when deploy it, i got this error:

java.lang.IllegalArgumentException: unknown setting [node.data_cold] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

Any idea please ?

Thank you in advance!

Answer 1

Assuming you're using the Elastic helm charts, I accomplished this by setting the following in my values.yml:

extraEnvs:
  - name: 'node.attr.data'
    value: '{{ ilm_phase }}'

and setting the following in my vars.yml for each individual data tier:

ilm_phase: 'cold'  # ...or hot, or whatever...

And then finally, using a custom node attribute in my ILM policy.

It's not ideal, but it works well, even if it's not as nuanced as using node.roles. If someone else has a better method, I'm open to it.

Edit

I forgot that I also added the following template, which applies to all new indices created. This forces all new indices to be created on the hot data nodes.

PUT _template/ilm-set-index-ilm-hot
{
  "order": 127,
  "index_patterns": [ "*" ],
  "settings": {
    "index": {
      "routing": {
        "allocation": {
          "require": {
            "data": "hot"
          }
        }
      }
    }
  },
  "mappings": {},
  "aliases": {}
}