Search code examples
amazon-web-servicesamazon-cloudwatchaws-cloudwatch-log-insights

Parsing JSON with CloudWatch Insight Logs


I am having difficulty in parsing my JSON to only show the ingestId of my messages. My JSON file that is uploaded to CloudWatch is like so:

{
    "message": "changeStatus ingestId=2343d8sf-etc, 
             status=UPLOADING",
    "level": "info"
}

My CloudWatch Log Insight code is like so:

filter level = "error"
| filter @message like /([-\w]{25,})/
| filter strcontains(@logStream, 'ingest-')
| fields @timestamp, @message, @logStream, level
| sort @logStream, @timestamp asc

Insight produces a new column with no information of the ID I would want. What am I doing wrong so I can at least the the ID of the ingest?

Would appreciate any help.

EDIT (v2): I was able to find the regex needed to get the Ingest ID with this regex code:

/([-\w]{25,})/

Here is the code. It successfully runs with a new column of ingestId but still no ingest ID that I am looking for:

enter image description here

filter level = "error"
| filter strcontains(@logStream, 'ingest-')
| filter @message like /(ingestId)/
| parse @message "\"ingestId\": \"/([-\w]{25,})/\"" as ingestId
| fields @timestamp, @message, @logStream, level
| sort @logStream, @timestamp asc

What can be done so I can display the ingest Id of each message coming in? Thanks all.


Solution

  • I was able to fix the issue to display the ingest Id in the column. I hope this will help anyone who is having troubles with this to be able to get information from your message:

    filter level = "error"
    | filter strcontains(@logStream, 'ingest-')
    | parse message /ingestId=(?<ingestId>[-\w]+)/
    | display ingestId
    | fields @timestamp, message, level, @logStream
    | sort @logStream, @timestamp asc