Trying to filter some Elasticsearch results where the field might not exist

Question

I have some data and I'm trying to add an extra filter that will exclude/filter-out any results which is where the key/value is foo.IsMarried == true.

Now, there's heaps of documents that don't have this field. If the field doesn't exist, then I'm assuming that the value is foo.IsMarried = false .. so those documents will be included in the result set.

Can anyone provide any clues, please?

I'm also using the .NET 'NEST' nuget client library - so I'll be really appreciative if the answer could be targeting that, but just happy with any answer, really.

Answer 1

Generally, within elasticsearch, for a boolean field, if the field doesn't exist, it doesn't mean that it's value is false. It could be that there is no value against it.

But, based on the assumption you are making in this case - we can check if the field foo.isMarried is explicitly false OR it does not exist in the document itself. The query presented by Rahul in the other answer does the job. However since you wanted a NEST version of the same, the query can be constructed using the below snippet of code.

// Notice the use of not exists here. If you do not want to check for the 'false' value,
// you can omit the first term filter here. 'T' is the type to which you are mapping your index.
// You should pass the field based on the structure of 'T'.
private static QueryContainer BuildNotExistsQuery()
{
    var boolQuery = new QueryContainerDescriptor<T>().Bool(
        b => b.Should(
            s => s.Term(t => t.Field(f => f.foo.IsMarried).Value(false)),
            s => !s.Exists(ne => ne.Field(f => f.foo.IsMarried))
        )
    );
}

You can trigger the search through the NEST client within your project as shown below.

var result = client.Search<T>(    
    .From(0)
    .Size(20)
    .Query(q => BuildNotExistsQuery())
    // other methods that you want to chain go here
)