Changing mapping fields structure flow in Elasticsearch

Question

I have an index with the mappings

{
   "mappings": {
      "properties": {
         "title": {
            "type": "text"
         },
         "location": {
            "type": "keyword"
         }
      }
   }
}

In location field at the moment we are storing the city name. And we need to change the mapping structure to store also the country and state, so the mapping will be

{
   "mappings": {
      "properties": {
         "title": {
            "type": "text"
         },
         "location": {
            "properties": {
                "country": {
                    "type": "keyword"
                },
                "state": {
                    "type": "keyword"
                },
                "city": {
                    "type": "keyword"
                }
            }
         }
      }
   }
}

What is the recommended flow for such migration?

Answer 1

Elasticsearch does not allow changing the definition of mapping for existing fields, just the addition of new field definitions as you can check here.

So one of the possibilities is:

• create a new field definition, with a different name obviously, to store the new data type.

• Stop to use the location field

The another but costly possibility is:

• create a new index with the right mapping
• do the reindex of the data from the old index to the new index

To reindex the data from the old index with the right format to the new index you can use a painless script:

POST /_reindex
{
  "source": {
    "index": "old_index_name"
  },
  "dest": {
    "index": "new_index_name"
  },
 "script": {
    "lang": "painless",
     "params" : {
         "location":{
            "country" : null,
             "state": null,
             "city": null
     }
    },
    "source": """
        params.location.city = ctx._source.location
        ctx._source.location = params.location
        
      """
    }
}

After you can update country and state fields for the old data.

If you need the same index name, use the new index you created with the correct mapping just as a backup, then you need to delete the index with the old mapping and recreate it again with the same name using the correct mapping and bring the data that are in the other reserve index.

For more about change the mapping read CHANGE ELASTIC SEARCH MAPPING.