Redistributing aws elasticsearch primary shard content into two equal sized shard in same existing index

Question

I need a way that whenever shard size increases from a given size limit, I need to redistribute that shard's data into two equal-size shards by adding one more shard and transfer half of the original size exceeded shard's data into newly created shard in the same index.

I have got the shard state like following, but need help find a way to distribute the data

{
    "index": "public",  
    "shard": "0",   
    "store": "20GB" 
}

P.S. I have tried Split Index API Link but this doesn't serve the purpose as it requires a new non-existing index and it cannot do the magic on the existing index, like in the above example index 'public' need to be the same but shard should increase and distribute data among themselves

Answer 1

This is not possible, you can't change the primary shards of elasticsearch index on the same index, this is because if your routing and location depend on the number of primary shards(created at the index creation time).

And if you change it, elasticsearch will have to change the routing algorithm and distribute the data again to evenly distribute the data in all the shards(including replica). Doing the above on a distributed large-scale stateful application is not an easy feat and elasticsearch as of now doesn't support it.