Elasticsearch Query to Retrieve a Fixed Number of Documents from Each Index

Question

I have an Elasticsearch setup with multiple indexes (let's say a, b, c, d, e, and f). My goal is to retrieve a total of 20 documents, with exactly 4 documents from each index. However, if any index has fewer than 4 documents, I want to fill in the remaining documents from other indexes.

For example:

a = 4
b = 4
c = 4
d = 4
e = 4
Total = 20

If any index has less than 4 documents:

a = 3
b = 4 or 5
c = 4 or 5
d = 4 or 5
e = 4 or 5
Total = 20

What should be the Elasticsearch query or payload to achieve this?

Current approach:

{
    "index": [
        "_all"
    ],
    "body": {
        "query": {
            "match_all": {}
        },
        "size": 20,
        "sort": {
            "created_at": "desc"
        }
    }
}

Any help or guidance is appreciated.

Thanks in advance!

Answer 1

I would probably use the multi-search API to do this and query each index separately, like this:

POST /_msearch
{"index": "a"}
{"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
{"index": "b"}
{"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
{"index": "c"}
{"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
{"index": "d"}
{"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}
{"index": "e"}
{"size": 10, "sort": {"created_at": "desc"}, "query" : {"match_all" : {}}}

Also note that I'm retrieving more than 4 hits per index, for the case where an index would have less than 4 hits, you can pick another hit from another index.