ElasticSearch DSL Matching all elements of query in list of list of strings

Question

I'm trying to query ElasticSearch to match every document that in a list of list contains all the values requested, but I can't seem to find the perfect query.

Mapping:

        "id" : {
          "type" : "keyword"
        },
        "mainlist" : {
          "properties" : {
            "format" : {
              "type" : "keyword"
            },
            "tags" : {
              "type" : "keyword"
            }
          }
        },
        ...

Documents:

doc1 {
   "id" : "abc",
   "mainlist" : [
            {
              "type" : "big",
              "tags" : [
                "tag1",
                "tag2"
              ]
             },
             {
              "type" : "small",
              "tags" : [
                "tag1"
              ]
             }
    ]
},
doc2 {
   "id" : "abc",
   "mainlist" : [
            {
              "type" : "big",
              "tags" : [
                "tag1"
              ]
             },
            {
              "type" : "small",
              "tags" : [
                "tag2"
              ]
             }
    ]
},
doc3 {
   "id" : "abc",
   "mainlist" : [
            {
              "type" : "big",
              "tags" : [
                "tag1"
              ]
             }
    ]
}

The query I've tried that got me closest to the result is:

GET /index/_doc/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "mainlist.tags": "tag1"
          }
        },
        {
          "term": {
            "mainlist.tags": "tag2"
          }
        }
      ]
    }
  }
}

although I get as result doc1 and doc2, while I'd only want doc1 as contains tag1 and tag2 in a single list element and not spread across both sublists.

How would I be able to achieve that? Thanks for any help.

Answer 1

As mentioned by @caster, you need to use the nested data type and query as in normal way Elasticsearch treats them as object and relation between the elements are lost, as explained in offical doc.

You need to change both mapping and query to achieve the desired output as shown below.

Index mapping

{
    "mappings": {
        "properties": {
            "id": {
                "type": "keyword"
            },
            "mainlist" :{
                "type" : "nested"
            }
        }
    }
}

Sample Index doc according to your example, no change there

Query

{
    "query": {
        "nested": {
            "path": "mainlist",
            "query": {
                "bool": {
                    "must": [
                        {
                            "term": {
                                "mainlist.tags": "tag1"
                            }
                        },
                        {
                            "match": {
                                "mainlist.tags": "tag2"
                            }
                        }
                    ]
                }
            }
        }
    }
}

And result

hits": [
            {
                "_index": "71519931_new",
                "_id": "1",
                "_score": 0.9139043,
                "_source": {
                    "id": "abc",
                    "mainlist": [
                        {
                            "type": "big",
                            "tags": [
                                "tag1",
                                "tag2"
                            ]
                        },
                        {
                            "type": "small",
                            "tags": [
                                "tag1"
                            ]
                        }
                    ]
                }
            }
        ]