Search code examples
elasticsearchfluentdefk

Disable mapping for a specific field using an Index Template Elasticsearch 6.8


I have an EFK pipeline set up. Everyday a new index is created using the logstash-* prefix. Every time a new field is sent by Fluentd, the field is added to the index pattern logstash-*. I'm trying to create an index template that will disable indexing on a specific field when an index is created. I got this to work in ES 7.1 using the PUT below:

PUT _template/logstash-test
{
  "index_patterns": ["logstash-*"],
  "mappings": {
    "dynamic_templates" : [
        {
          "params" : {
            "path_match" : "params",
            "mapping" : {
              "enabled": false
            }
          }
        }
      ]
  }
}

However when I try this on Elasticsearch 6.8 I get the following error:

"type": "illegal_argument_exception",
"reason": "Malformed [mappings] section for type [dynamic_templates], should include an inner object describing the mapping"

Solution

  • It is a little different in Elasticsearch 6.X as it had mapping types, which is not used anymore.

    Try something like this:

    PUT _template/logstash-test
    {
        "index_patterns": ["logstash-*"],
        "mappings": {
            "_doc": {
                "dynamic_templates" : [
                    {
                        "params" : {
                            "path_match" : "params",
                            "mapping" : {
                                "enabled": false
                            }
                        }
                    }
                ]
            }
        }
    }
    

    If your index has a different custom type and is not using the _doc type, you should use that in the mapping.