Search code examples
elasticsearchfilterquerydsl

Why does 'exists' fail for normal queries and not for filtered queries?


Consider the following two queries :

{
  "_source" : false,
  "query": {
    "bool": {
      "must": [
        {
          "range": {
            "date": {
              "gte": 1460789348000,
              "lte": 1466059748000
            }
          }
        }
      ]
    }
  }
}

Now consider the following query with filtered and filter fields added :

{
  "_source": false,
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "date": {
                  "gte": 1460789348000,
                  "lte": 1466059748000
                }
              }
            }
          ]
        }
      }
    }
  }
}

Both give the same result and the 1st query without the filtered and filter fields doesn't result in an error unlike the following query :

{
  "_source": false,
  "query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "comment"
          }
        }
      ]
    }
  }
}

Error:

{
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[bi8BrruxR9i_H8-n4Qo9Mg][acct_prop_5][0]: RemoteTransportException[[Battletide][inet[/172.16.0.11:9302]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[acct_prop_5][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[acct_prop_5] No query registered for [exists]]; }{[JorHtifCSxGrbb3ovGJMWQ][advocacy_insights_p5_v1_00000000_0000][0]: RemoteTransportException[[Caiera][inet[/172.16.0.11:9304]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[advocacy_insights_p5_v1_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[advocacy_insights_p5_v1_00000000_0000] No query registered for [exists]]; }{[Tlt3tXF_S9-rEmK9iwSvoA][audience_act_p5_v2_00000000_0000][0]: RemoteTransportException[[Monet St. Croix][inet[/172.16.0.11:9303]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[audience_act_p5_v2_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[audience_act_p5_v2_00000000_0000] No query registered for [exists]]; }{[NTpZwKeYRI6l6zHf2BlIFg][audience_act_p5_v3_00000000_0000][0]: SearchParseException[[audience_act_p5_v3_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[audience_act_p5_v3_00000000_0000] No query registered for [exists]]; }{[NTpZwKeYRI6l6zHf2BlIFg][audience_act_p5_v3_20150601_0000][0]: SearchParseException[[audience_act_p5_v3_20150601_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[audience_act_p5_v3_20150601_0000] No query registered for [exists]]; }{[Tlt3tXF_S9-rEmK9iwSvoA][auth2][0]: RemoteTransportException[[Monet St. Croix][inet[/172.16.0.11:9303]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[auth2][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[auth2] No query registered for [exists]]; }{[JorHtifCSxGrbb3ovGJMWQ][authors][0]: RemoteTransportException[[Caiera][inet[/172.16.0.11:9304]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[authors][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[authors] No query registered for [exists]]; }{[bi8BrruxR9i_H8-n4Qo9Mg][bm_content_lifetime_p5_v1_00000000_0000][0]: RemoteTransportException[[Battletide][inet[/172.16.0.11:9302]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[bm_content_lifetime_p5_v1_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[bm_content_lifetime_p5_v1_00000000_0000] No query registered for [exists]]; }{[NTpZwKeYRI6l6zHf2BlIFg][bm_content_lifetime_p5_v2_00000000_0000][0]: SearchParseException[[bm_content_lifetime_p5_v2_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: QueryParsingException[[bm_content_lifetime_p5_v2_00000000_0000] No query registered for [exists]]; }{[bi8BrruxR9i_H8-n4Qo9Mg][bm_content_lifetime_p5_v93_00000000_0000][0]: RemoteTransportException[[Battletide][inet[/172.16.0.11:9302]][indices:data/read/search[phase/query]]]; nested: SearchParseException[[bm_content_lifetime_p5_v93_00000000_0000][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"_source":false,"query":{"bool":{"must":[{"exists":{"field":"comment"}}]}}}]]]; nested: .........

Whereas with the two fields it becomes a correct query:

{
  "_source": false,
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "exists": {
                "field": "comment"
              }
            }
          ]
        }
      }
    }
  }
}

What is the difference between range and exists query which allows only one query to be valid without the filtered and filter fields?


Solution

  • You probably must be using an older version of Elasticsearch [version < 2.0] where exists is a filter. There is no exists query which is why you are getting the error message:No query registered for [exists].

    Instead you may use:

    {
       "_source": false,
       "filter": {
          "bool": {
             "must": [
                {
                   "exists": {
                      "field": "comment"
                   }
                }
             ]
          }
       }
    }
    

    Or simply:

    {
       "_source": false,
       "filter": {
          "exists": {
             "field": "comment"
          }
       }
    }
    

    From Elasticsearch version 2.0 onwards, the query and filter objects have been merged together and there exists only query object which is capable of doing both the functionalities. Hence you can use exists query from version 2.0 onwards.

    You can read more about the merge at: Better query execution coming to Elasticsearch 2.0