Elastic Search 2.0/2.1 Issue with Highlighter and the Bool Query

Question

I am having an issue with highlighting in Elastic 2.0 and 2.1 - it's returning more information than I think it should.

I am constructing a bool query (the filtered query keyword is deprecated in 2.0+ so I am trying to update my syntax). I am building a must section and a filter section within the query, followed by a request for highlighting information.

The documentation says to use the query either in a query context or a filter context, but the highlighter doesn't seem to denote such a distinction.

Here is my fully formed query:

GET /sample04/_search
{
    "query": {
        "bool": {
            "must": [
                {
                    "query": { "query_string": { "query": "east west" } }
                }
            ],
            "filter": [
                {
                    "terms": {"OwnerId": ["1", "2","3"]}
                }
            ]
        }
    },
    "highlight": {
        "fields": {
            "*": { "require_field_match": "false" }
        }
    }
}

So this query works as expected - we are querying for terms east or west, and we are filtering documents on an Id field that is part of our security requirements, and then I ask for highlighting information.

The downside, however, is the highlighting information contains a hit every instance of every value I submitted in my filter (in this case 1, 2 or 3) that matched any value in any field in any part of my document, like this:

 "highlight": {
               "SomeTextField": [
                  "North <em>West</em>"
               ],
               "OwnerId": [
                  "<em>3</em>"
               ],
               "SerialNumber": [
                  "<em>3</em>-<em>3</em>"
               ],
               "AssociatedValue": [
                  "<em>3</em>",
                  "<em>2</em>"
               ],
               "RelatedValue": [
                  "<em>3</em>",
                  "<em>3</em>",
                  "<em>3</em>",
                  "<em>3</em>",
                  "<em>3</em>"
               ]
            }

How do I get the highlighter to match my query in the must section, but ignore the filter? It is my belief that it should ignore highlighting matches that were part of the filter, notably when it's highlighting fields that contain values were requested to filter a SPECIFIC FIELD, but it's utilizing the value anywhere within my document. This seems wrong somehow, but perhaps it's my understanding.

As an FYI, if I set require_field_match to TRUE, then I ONLY get hits that match the filter, and NONE that match the query.

I cannot specify a field to generate highlighting information for, whereas we consume Elastic as a search once find anywhere model, so I don't know field my result will return from.

Can you see what I'm doing wrong? It would be greatly appreciated to understand this.

Answer 1

You can use highlight query for this purpose. change your highlight part to

"highlight": {
    "fields": {
      "*": {
        "highlight_query": {
          "query_string": {
            "query": "east west"
          }
        }
      }
    }
  }