Lambda InvalidParameterException while trying to filter Cloudwatch Logs

Question

I am trying to create a Lambda funtion to search cloudwatch logs for a string. If there are 5 occurrences of that value within 10 minutes send an email or sns.

I am not sure if the error is pointing me in the right direction: Error when I test it:

  {
   "errorMessage": "An error occurred (InvalidParameterException) when calling the FilterLogEvents operation: Invalid character(s) in term '<'",
     "errorType": "InvalidParameterException",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 24, in lambda_handler\n    response = cloudwatch.filter_log_events(\n",
    "  File \"/var/runtime/botocore/client.py\", line 391, in _api_call\n    return   self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 719, in _make_api_call\n    raise    error_class(parsed_response, operation_name)\n"
   ]
  }

I have created a IAM role to use the Lambda Service and it has filter logs allow.

any help is greatly appreciated:

import json

def lambda_handler(event, context):
# TODO implement
return {
    'statusCode': 200,
    'body': json.dumps('Hello from Lambda!')
}
import boto3
import datetime

def lambda_handler(event, context):
log_group_name = 'security'  #  log group name
search_string = '4625'   # string being searched
sns_topic_arn = 'arn:aws-us-yadyaxxx   #  SNS topic ARN
custom_email = 'myemailaddy@.com'   # custom email

cloudwatch = boto3.client('logs')
sns = boto3.client('sns')

end_time = int(datetime.datetime.now().timestamp())
start_time = end_time - 600  # 600 seconds = 10 minutes

response = cloudwatch.filter_log_events(
    logGroupName=log_group_name,
    startTime=start_time,
    endTime=end_time,
    filterPattern=search_string
)

occurrences = sum(1 for _ in response['events'])

if occurrences >= 5:
    message = f"Found '{search_string}' 5 times within 10 minutes in {log_group_name}."

    sns.publish(
        TopicArn=sns_topic_arn,
        Message=message
    )

    sns.publish(
        TopicArn=sns_topic_arn,
        Message=message,
        Subject='Critical Log Events Detected',
        MessageAttributes={
            'email': {
                'DataType': 'String',
                'StringValue': custom_email
            }
        }
    )

I have checked the IAM role and it has filter logs allowed also tried various search patterns; I dont think thats it.

Answer 1

"An error occurred (InvalidParameterException) when calling the FilterLogEvents operation: Invalid character(s) in term '<'"

You're using special characters in your filter that are not allowed, in this case <.

I ran the code you posted and was able to reproduce the error by using search_string = '123<123

See Filter pattern syntax for supported characters and regex syntax.