Search code examples
performanceelasticsearchmodel-associations

How does the indexing and retrieving performance degrade with increase in data size in elastisearch Service?


I would like to know how the performance of the ElasticSearch service decreases/degrades with in size of the data increasing. Are there any metrics which indicates this?


Solution

  • It depends on

    • number of nodes in you cluster (more power and ram, but more time on communication between nodes)
    • document structure (complex structure longer to process)
    • indexing settings and mappings
    • volume of indexing/search queries (big volume of indexing query could slowdown search or oposite)
    • hdd speed for databases not fit in memory
    • type of search queries (simple CRUD queries, document search, aggregation and analytics)

    I would suggest you to start with ElasticHQ and list of diagnostic metrics

    enter image description here

    and Marvel from Elasticsearch

    It will give you an idea of potential bottlenecks

    Fill free to ask more specific questions when you have them

    ElasticSearch Usage Statistics from elasticsearch hq plugin

    http://www.elastichq.org/elasticsearchstats.php