Search code examples
solrshardingdistributed-system

Solr shard distribution data not distributed evenly


I am trying to build Solr shard set up with 8 shards. I tried composite key using year as distribution index.
The issue i found after running multiple configuration that it gives me highly uneven distribution. My shard key is to divide one year data on two shards.
But when i create data from year 2001 to 2017 some shard are taking 6 year data and some are taking only one year data.
I have added current distribution here. I run it multiple times but every time i got similar type of distribution.
While inserting documents i insert records one time for a particular year like all data of 2017 is ingested then 2016.
Can anyone help me how this is working. as each shard has 16 GB of memory and in this distribution two shard already have data more than 17 GB.

enter image description here


Solution

  • It worked with reduced time period. We did it for a quarter.