Search code examples
elasticsearchelasticsearch-aggregation

How pagination works in elasticsearch


Can you please tell me how pagination works in case I do not specify any "sort" field?

In my query, if I do not specify "sort" field and then try pagination using "from" plus "size", how does elasticsearch return data? is there any chance of returning duplicated records?

es query:

GET index1, index2/_search?pretty
{
  "from": 10,
  "size": 20,
  "_source": {"includes”:[“name”, “type”]}, 
   "query":{
      "bool":{
         "must":[
            {
               "term":{
                  "org_id":{
                     "value”:”1234456767257”,
                     "boost":1.0
                  }
               }
            },
            {
        "range" : {
          “Timestamp” : {
            "from" : 1596545417000,
            "to" : 1597035223465,
            "include_lower" : true,
            "include_upper" : true,
            "boost" : 1.0
          }
        }
      }
         ],
         "adjust_pure_negative":true,
         "boost":1.0
      }
   }
}

In this above query, I did not specify any "sort" field. Will this ensure of returning unique results if I increase "from" after each call? If Yes, HOW?


Solution

  • By default, the documents are sorted by _score:desc, that's probably not what you want if you're constantly indexing new documents.

    I would definitely add a sort clause on _doc, that's the only way to guarantee you're going to get unique results.

    {
      "from": 10,
      "size": 20,
      "sort": [
        {
          "_score": "desc"     <--- add this
        },
        {
          "_doc": "asc"        <--- and this
        }
      ],
      ...
    }
    

    Also if you consider paginating beyond from: 10000, you need to leverage search_after