Query hashmap structure with elasticsearch

Question

I have two questions regarding mapping and querying a java hashmap in elasticsearch.

Does this mapping make sense in elasticsearch (is it the correct way to map a hashmap)?:

{
  "properties": {
    "itemsMap": {
      "type": "nested",
      "properties": {
        "key": {
          "type": "date",
          "format": "yyyy-MM-dd"
        },
        "value": {
          "type": "nested",
          "properties": {
            "itemVal1": {
              "type": "double"
            },
            "itemVal2": {
              "type": "double"
            }
          }
        }
      }
    }
  }
}

Here is some example data:

{
  "itemsMap": {
    "2021-12-31": {
      "itemVal1": 100.0,
      "itemVal2": 150.0,
    },
    "2021-11-30": {
      "itemVal1": 200.0,
      "itemVal2": 50.0,
    }
  }
}

My queries don't seem to work. For example:

{
  "query": {
    "nested": {
      "path": "itemsMap",
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "itemsMap.key": "2021-11-30"
              }
            }
          ]
        }
      }
    }
  }
}

Am I doing something wrong? How can I query such a structure? I have the possibility to change the mapping if it's necessary.

Thanks

Answer 1

TLDR;

The way you are uploading your data, nothing is stored in key. You will have fields named 2021-11-30 ... and key is going to be empty.

Either you have a limited amount of "dates" and this is a viable options (less than 1000) else your format is not viable on the long run.

If you don't want to change your doc, here is the query

GET /71525899/_search
{
  "query": {
    "nested": {
      "path": "itemsMap",
      "query": {
        "bool": {
          "must": [
            {
              "exists": {
                "field": "itemsMap.2021-12-31"
              }
            }
          ]
        }
      }
    }
  }
}

To understand

If you inspect the mapping by querying the index

GET /<index_name>/_mapping

You will see that the number of fields name after your date is going to grow.

And in all your doc, itemsMap.key is going to be empty. (this explain why my previous answer did not work.

A more viable option

Keep your mapping, update the shape of your docs.

They will look like

{
  "itemsMap": [
    { 
      "key": "2021-12-31", 
      "value": { "itemVal1": 100, "itemVal2": 150 }
    },
    { 
      "key": "2021-11-30",
      "value": { "itemVal1": 200, "itemVal2": 50 }
    }
  ]
}

DELETE /71525899


PUT /71525899/
{
  "mappings": {
    "properties": {
      "itemsMap": {
        "type": "nested",
        "properties": {
          "key": {
            "type": "date",
            "format": "yyyy-MM-dd"
          },
          "value": {
            "type": "nested",
            "properties": {
              "itemVal1": {
                "type": "double"
              },
              "itemVal2": {
                "type": "double"
              }
            }
          }
        }
      }
    }
  }
}

POST /_bulk
{"index":{"_index":"71525899"}}
{"itemsMap":[{"key":"2021-12-31", "value": {"itemVal1":100,"itemVal2":150}},{"key":"2021-11-30", "value":{"itemVal1":200,"itemVal2":50}}]}
{"index":{"_index":"71525899"}}
{"itemsMap":[{"key":"2022-12-31", "value": {"itemVal1":100,"itemVal2":150}},{"key":"2021-11-30", "value":{"itemVal1":200,"itemVal2":50}}]}
{"index":{"_index":"71525899"}}
{"itemsMap":[{"key":"2021-11-31", "value": {"itemVal1":100,"itemVal2":150}},{"key":"2021-11-30", "value":{"itemVal1":200,"itemVal2":50}}]}



GET /71525899/_search
{
  "query": {
    "nested": {
      "path": "itemsMap",
      "query": {
        "bool": {
          "must": [
            {
              "match": {
                "itemsMap.key": "2021-12-31"
              }
            }
          ]
        }
      }
    }
  }
}