Elasticsearch compound query

Question

I am querying an elastic index with 300 records using the compound query like below:

GET my_index/_search
{
  "size": 10,
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "should": [
              {
                "multi_match": {
                  "query": "card",
                  "fields": [
                    "title^1.0"
                  ]
                }
              }
            ],
            "must": {
              "term": {
                  "_index": {
                    "value": "my_index"
                  }
                }
            }
          }
        }
      ]
    }
  }
}

The must on index is because this could be a multi index query depending on some business logic (must should most probably be a filter, and I can change that but that is not the part of my question. I get the same results with filter as well).

While I expect this to return the documents that match the should clause, I am getting back all the documents in the index (300)

Why would this happen?

Answer 1

The resolution to this was to add minimumShouldMatch field to the query. The resultant query then becomes:

GET my_index/_search
{
  "size": 10,
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "minimum_should_match": 1,
            "should": [
              {
                "multi_match": {
                  "query": "card",
                  "fields": [
                    "title^1.0"
                  ]
                }
              }
            ],
            "must": {
              "term": {
                  "_index": {
                    "value": "my_index"
                  }
                }
            }
          }
        }
      ]
    }
  }
}

The reasoning behind this I believe is that bool query is tuned to give maximum number of matched results (more-matches-is-better). So if must/filter clause matches then should is not even executed. By adding "minimum_should_match": 1 we are instructing elasticsearch to match at least 1 should clause before returning the document.

Excerpts from elastic documentation:

The bool query takes a more-matches-is-better approach, so the score from each matching must or should clause will be added together to provide the final _score for each document.

You can use the minimum_should_match parameter to specify the number or percentage of should clauses returned documents must match.

If the bool query includes at least one should clause and no must or filter clauses, the default value is 1. Otherwise, the default value is 0.

For other valid values, see the minimum_should_match parameter.

Link for reference - https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html#bool-min-should-match