Search code examples
elasticsearchelasticsearch-query

ES returns only 10 records by default. How to get all records without using scroll API

When we query ES for records it returns 10 records by default. How can I get all the records in the same query without using any scroll API.

There is an option to specify the size, but size is not known in advance.

Solution

  • You can retrieve up to 10k results in one request (setting "size": 10000). If you have less than 10k matching documents then you can paginate over them using a combination of from/size parameters. If it is more, then you would have to use other methods:

    Note that from + size can not be more than the index.max_result_window index setting which defaults to 10,000. See the Scroll or Search After API for more efficient ways to do deep scrolling.

    To be able to do pagination over unknown number of documents you will have to get the total count from the first returned query.

    Note that if there are concurrent changes in the data, results of paginated retrieval may not be consistent (for example, if one document gets inserted or deleted while you are paginating). Scroll is consistent since it is created from a "snapshot" of the index created at query start time.