Search code examples
elasticsearchelastic-stack

Disable dynamic mapping completely in Elasticsearch


I have an index template, from which I am creating an index

PUT /_index_template/example_template
{
   "index_patterns": [
        "example*"
    ],
    "priority": 1,
    "template": {
        "aliases": {
            "example":{}
        },
    "mappings": {
"dynamic":strict,
      "_source":
      {"enabled": false},
      "properties": {
       "SomeID":
        { "type": "keyword", "index" : true,"store":true,"ignore_above":5},
       "firstName":
        { "type": "text", "index" : true,"store":true},
        "lastName":
        { "type": "text", "index" : false},
        "PersonInfo": {
        "type": "object",
"dynamic":"true",
        "properties": {
          "FirstName": {
            "type": "keyword",
            "index": true,
            "store": false
          }
        }
        }
      }
    },
    "settings": {
    "index": {
      "number_of_shards": 1,  
      "number_of_replicas": 3
    }
  }
  }
}

As in the template mappings you can see I am making the dynamic as Strict, so that new fields cant be added to the mappings, while on inner object, PersonInfo, I can set dynamic as true, which takes precedence and allow to insert a new field mapping.

PUT example10022021/_doc/1
{
   "SomeID":"1234",
   "firstName":"Nishikant",
   "PersonInfo.service_data":"random"
}

Here service_data is getting added into mappings, as dynamic is true

"PersonInfo" : {
          "dynamic" : "true",
          "properties" : {
            "FirstName" : {
              "type" : "keyword"
            },
            "service_data" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }

Is there any way to disable the dynamic mapping completely? like specifying globally?

Thanks!

Steps I took After @Val answer:

PUT /_index_template/example_template
{
   "index_patterns": [
        "example*"
    ],
    "priority": 1,
    "template": {
        "aliases": {
            "order":{}
        },
    "mappings": {
      "dynamic": "strict",
      "dynamic_templates": [
        {
          "objects": {
            "match_mapping_type": "object",
            "mapping": {
              "dynamic": "strict"
            }
          }
        }
      ],
      "_source":
      {"enabled": false},
      "properties": {
       "BillToID":
        { "type": "keyword", "index" : true,"store":true,"ignore_above":5},
       "firstName":
        { "type": "text", "index" : true,"store":true},
        "lastName":
        { "type": "text", "index" : false},
        "PersonInfo": {
        "type": "object",
        "dynamic":true,
        "properties": {
          "FirstName": {
            "type": "keyword",
            "index": true,
            "store": false
          }
        }
        }
      }
    },
    "settings": {
    "index": {
      "number_of_shards": 1,  
      "number_of_replicas": 3
    }
  }
  }
}

then I create an index

PUT example10022021

then inserting a document

POST example10022021/_doc/1
{
   "BillToID":"1234",
   "firstName":"Nishikant",
   "PersonInfo.service_data":"random"
}

this will result in 200OK, now if you check the mappings again

GET example10022021

in o/p you can see the dynamic field mapping getting added(this behavior I don't want),

"PersonInfo" : {
          "dynamic" : "true",
          "properties" : {
            "FirstName" : {
              "type" : "keyword"
            },
            "service_data" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            }
          }
        }

Solution

  • What you can do is to create another index template that applies to all indexes, i.e. using the * name pattern:

    PUT /_index_template/common_template
    {
       "index_patterns": [
            "*"
        ],
        "priority": 0,
        "template": {
           "mappings": {
             "dynamic": "strict",
             ...
    

    If you want to also restrict the creation of dynamic fields inside inner objects, you can leverage dynamic templates, like this:

    PUT /_index_template/common_template
    {
      "index_patterns": [
        "*"
      ],
      "priority": 1000,
      "template": {
        "settings": {},
        "mappings": {
          "dynamic": "strict",
          "dynamic_templates": [
            {
              "objects": {
                "match_mapping_type": "object",
                "mapping": {
                  "dynamic": "strict"
                }
              }
            }
          ],
          "properties": {
            "test": {
              "type": "object",
              "properties": {
                "inner": {
                  "type": "integer"
                }
              }
            }
          }
        }
      }
    }
    

    With the above index template, you can create a document like this one:

    POST test/_doc/
    {
      "test": {
        "inner": 1
      }
    }
    

    But not like this one:

    POST test/_doc/
    {
      "test": {
        "inner": 1,
        "inner2": 2        <--- this will not be allowed
      }
    }