Search code examples
amazon-web-servicesaws-glueaws-event-bridge

Using an EventBridge event pattern string in a lambda function


I have a lambda function using Python. It's connected to an EventBridge rule that triggers every time there's a change in a Glue table.

The event pattern it's outputting looks something like this:

{
    "version":"0",
    "detail":{
        "databaseName":"flights-db",
        "typeOfChange":"UpdateTable",
        "tableName":"flightscsv"
    }
}

I want to get the tableName and databaseName values from this output into the function as a variable.

My Lambda function:

import json
import boto3

def lambda_handler(event, context):
    boto3_version = boto3.__version__

    return_statement = 'Boto3 version: ', boto3_version,\
                       'Event output: ', event

    return {
        'statusCode': 200,
        'body': json.dumps(return_statement)
    }

I was expecting to get the event pattern output from the event in my return statement but that's not the case.

When testing this function the return output for event is:

{\"key1\": \"value1\", \"key2\": \"value2\", \"key3\": \"value3\"}

This key and values are defined like this in the Test Pattern for the function.

The eventbridge rule is defined like this: enter image description here

How can I get the values from the event pattern to a variable? Do I need to configure the test pattern to get the results into event?

EDIT: Picture of log events for the table change event: enter image description here


Solution

  • The event object generated by CloudWatch (CW) Events / Event Bridge (EB) are listed here. These events will be passed to your function when it is going to get triggered by EB.

    Your EB Event Pattern should be:

    {
      "source": ["aws.glue"],
      "detail-type": ["Glue Data Catalog Table State Change"]
    }
    

    The above should match changes to any tables in your glue catalog. The event should be similar to the one below:

    {
        "version": "0",
        "id": "2617428d-715f-edef-70b8-d210da0317a0",
        "detail-type": "Glue Data Catalog Table State Change",
        "source": "aws.glue",
        "account": "123456789012",
        "time": "2019-01-16T18:16:01Z",
        "region": "eu-west-1",
        "resources": [
            "arn:aws:glue:eu-west-1:123456789012:table/d1/t1"
        ],
        "detail": {
            "databaseName": "d1",
            "changedPartitions": [
                "[C.pdf, dir3]",
                "[D.doc, dir4]"
            ],
            "typeOfChange": "BatchCreatePartition",
            "tableName": "t1"
        }
    }
    

    Thus, to get tableName and databaseName your lambda function could be:

    import json
    import boto3
    
    def lambda_handler(event, context):
        boto3_version = boto3.__version__
    
        print(event)
    
        table_name = event['detail']['tableName']
        database_name = event['detail']['databaseName']
    
        print(table_name, database_name)
    
        return_statement = {
            'boto3_version': boto3_version,
            'table_name': table_name,
            'database_name': database_name
        }
    
        return {
            'statusCode': 200,
            'body': json.dumps(return_statement)
        }
    

    For testing, you can setup sample EB event in your lambda test window:

    enter image description here