Build Elasticsearch query dynamically by extracting fields to be matched from the Lambda event in AWS Elasticsearch service

Question

I want to write a query to match indexed fields in Elasticsearch. I am using AWS Elasticsearch service and writing the query as an AWS Lambda function. This lambda function is executed when an event occurs, searches for a fields sent in the event, matches the fields with the indexed documents and returns the matched documents.

However, we don't know the fields or the number of fields to be searched ahead of time. So I want to be able to extract the fields from the event in the lambda function and construct the query dynamically to match the fields with the indexed documents.

The event is as follows:

{
    "headers": {
      "Host": "***"
    },
    "queryStringParameters": {
      "fieldA": "abc",
      "fieldB": "def"
    }
  }

The lambda function is as follows. This function expects two fields and matches them.

def search(event, context):

    fields = list(event['queryStringParameters'].keys())
    firstField = fields[0]
    secondField = fields[1]
    
    values = list(event['queryStringParameters'].values())
    firstValue = values[0]
    secondValue = values[1]
    
    query = {
        "query": {
            "bool" : {
                "must" :[
                     {"match" : { firstField : firstValue }},
                     {"match" : { secondField : secondValue }}
                    ] 
            }
        }
    }

How can I rewrite my query so it dynamically accepts the fields and the number of fields that the event sends (not known ahead of time)?

Answer 1

Not sure what your exact requirements are but you could go with the following:

def search(event, context):
    query = {
        "query": {
            "bool": {
                "query_string": {
                    "query": " OR ".join([
                        "(%s:'%s')" % (k, v) for (k, v) in event["queryStringParameters"].items()
                    ])
                }
            }
        }
    }

    print(query)

which'd result in a proper query_string_query:

{
  "query":{
    "bool":{
      "query_string":{
        "query":"(fieldB:'def') OR (fieldA:'abc')"
      }
    }
  }
}

You could interchange the OR with an AND. Also keep in mind that when the values are wrapped in quotes, ES will enforce exact matches. Leave them out in case you're after a contains behavior (i.e. a match query).