Search code examples
elasticsearch

how to rename an index in a cluster?

I need to rename several indexes in a cluster (their name must be changed, I cannot use aliases).

I saw that there are no supported ways to do that, the closest I found is to rename the directory of the index, I tried this in a cluster.

The cluster has 3 machines A, B and C and the shards are replicated on each of them. I shut down elasticsearch on A, renamed /var/lib/elasticsearch/security/nodes/0/indices/oldindexname to /var/lib/elasticsearch/security/nodes/0/indices/newindexname and restarted A.

The state of the cluster was yellow and elasticsearch was doing some magic to restore a correct state. After some time I ended up with

  • oldindexname being available and fully replicated (recovered from B and C I guess)
  • newindexname being available (I can search it) but the head plugin shows that its shards are in an "Unassigned" state and that they are grayed out (not replicated)

During the recovery security.log showed the following message:

[2015-02-20 11:02:33,461][INFO ][gateway.local.state.meta ] [A.example.com] dangled index directory name is [newindexname], state name is [oldindexname], renaming to directory name

While newindexname is searchable, it is certainly not in a normal state.

I rolled back to the previous state by deleting newindexname. The cluster is back to green without any "Unassigned" entries.

Given that, how can I rename oldindexname to newindexname in a cluster?

Note: the ultimate solution I have in mind is to scroll-copy oldindex into newindex and delete oldindex afterwards. This is going to take time so if there is a more direct solution it would be great.

Solution

  • Starting with ElasticSearch 7.4, the best method to rename an index is to copy the index using the newly introduced Clone Index API, then to delete the original index using the Delete Index API.

    The main advantage of the Clone Index API over the use of the Snapshot API or the Reindex API for the same purpose is speed, since the Clone Index API hardlinks segments from the source index to the target index, without reprocessing any of its content (on filesystems that support hardlinks, obviously; otherwise, files are copied at the file system level, which is still much more efficient that the alternatives). Clone Index also guarantee that the target index is identical in every point to the source index (that is, there is no need to manually copy settings and mappings, contrary to the Reindex approach), and doesn't require a local snapshot directory be configured.

    Side note: even though this procedure is much faster than previous solutions, it still implies down time. There are real use cases that justify renaming indices (for example, as a step in a split, shrink or backup workflow), but renaming indices should not be part of day-to-day operations. If your workflow requires frequent index renaming, then you should consider using Indices Aliases instead.

    Here is an example of a complete sequence of operations to rename index source_index to target_index. It can be executed using some ElasticSearch specific console, such as the one integrated in Kibana. See this gist for an alternative version of this example, using curl instead of an Elastic Search console.

    # Make sure the source index is actually open
POST /source_index/_open

# Put the source index in read-only mode
PUT /source_index/_settings
{
  "settings": {
    "index.blocks.write": "true"
  }
}

# Clone the source index to the target name, and set the target to read-write mode
POST /source_index/_clone/target_index
{
  "settings": {
    "index.blocks.write": null 
  }
}

# Wait until the target index is green;
# it should usually be fast (assuming your filesystem supports hard links).
GET /_cluster/health/target_index?wait_for_status=green&timeout=30s

# If it appears to be taking too much time for the cluster to get back to green,
# the following requests might help you identify eventual outstanding issues (if any)
GET /_cat/indices/target_index
GET /_cat/recovery/target_index
GET /_cluster/allocation/explain

# Delete the source index
DELETE /source_index