Search code examples
elasticsearchkibanaelastic-stackelk

optimization on old indexes collecting logs from my apps


I have an elastic cluster with 3x nodes(each 6x cpu, 31GB heap , 64GB RAM) collecting 25GB logs per day , but after 3x months I realized my dashboards become very slow when checking stats in past weeks , please, advice if there is an option to improve the indexes read erformance so it become faster when calculating my dashboard stats?

Thanks!


Solution

  • I would suggest you try to increase the shards number when you have more shards Elasticsearch will split your data over the shards so as a result, Elastic will send multiple parallel requests to search in a smaller data stack

    for Shards number you could try to split it based on your heap memory size
    No matter what actual JVM heap size you have, the upper bound on the maximum shard count should be 20 shards per 1 GB of heap configured on the server.

    ElasticSearch - Optimal number of Shards per node https://qbox.io/blog/optimizing-elasticsearch-how-many-shards-per-index https://opster.com/elasticsearch-glossary/elasticsearch-choose-number-of-shards/