Search code examples
amazon-web-servicesaws-lambdaaws-cliamazon-cloudwatch

AWS - Cloudwatch Dashboard Deployment - CLI - 'Search Parameter'


Hope you're all doing well. I am reaching out to you because I am working on configuring a bash script that I can employ to dynamically deploy to AWS Cloudwatch Dashboards irrespective of AWS Instance.

So far, I'm having some success with it, but there is something that I am not getting my hands around. I'm hoping that I can get a little advice.

Since I want to be able to set these up through the CLI, and not through the portal, I don't know how to get the entire set of lambdas to deploy dynamically, except by setting up the widget manually. Now, as I understand it, I should be using the "Search" parameter in the 'expression' clause.

This is what I'm doing. Additionally, I am using that clause for other things in AWS (like for example, Connect Queue Wait Times) and it works successfully. However, in the case that I am attempting to use it for the Lambdas, it is not working. Now, I'm using the search clause for 3 other AWS Connect features, so my best guess is that I am using it incorrectly, or that it does not apply for the lambdas. Below I have attached two examples of how I am entering it.

AWS Connect Queue Wait Time

In this case, it is working.

"metrics": [[ { "expression": "SEARCH('{AWS/Connect,QueueName, InstanceId, MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]]

Lambda Errors - Widget - By Lambda

In this case it is not.

"metrics": [[ { "expression": "SEARCH('{AWS/Lambda,FunctionName, InstanceId, MetricName}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" }]]

Hoping that someone understands why the lambda (second) one isn't working, but the first is. I don't think it will be helpful, but below is the entire body of code.

{
    "start": "-PT3H",
    "periodOverride": "inherit",
    "widgets": [
        {
            "height": 6,
            "width": 6,
            "y": 6,
            "x": 0,
            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/Connect", "CallsPerInterval", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "visible": false } ],
                    [ ".", "ConcurrentCallsPercentage", ".", ".", ".", ".", { "visible": false } ],
                    [ ".", "ConcurrentCalls", ".", ".", ".", "." ],
                    [ ".", "MissedCalls", ".", ".", ".", ".", { "visible": false } ],
                    [ ".", "CallRecordingUploadError", ".", ".", ".", "CallRecordings", { "visible": false } ]
                ],
                "view": "timeSeries",
                "stacked": false,
                "region": "${instanceRegion}",
                "title": "Concurrent Calls - Widget",
                "period": 300,
                "yAxis": {
                    "left": {
                        "label": "miliseconds",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "stat": "Average",
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 120
                        }
                    ]
                }
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 0,
            "x": 6,
            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/Connect", "ToInstancePacketLossRate", "Participant", "Agent", "Type of Connection", "WebRTC", "Instance ID", "${instanceId}", "Stream Type", "Voice", { "period": 60, "stat": "Average", "color": "#2ca02c" } ],
                    [ "...", { "period": 60, "stat": "Maximum", "color": "#d62728" } ]
                ],
                "view": "timeSeries",
                "stacked": false,
                "region": "${instanceRegion}",
                "title": "Instance Packet Loss - Widget",
                "yAxis": {
                    "left": {
                        "label": "Packet",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "period": 300,
                "liveData": true,
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 0.8
                        }
                    ]
                }
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 0,
            "x": 12,
            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/Connect", "ConcurrentCallsPercentage", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "visible": false } ],
                    [ ".", "ConcurrentCalls", ".", ".", ".", ".", { "visible": false } ],
                    [ ".", "CallsPerInterval", ".", ".", ".", ".", { "visible": false } ],
                    [ ".", "MissedCalls", ".", ".", ".", "." ],
                    [ ".", "CallRecordingUploadError", ".", ".", ".", "CallRecordings", { "visible": false } ]
                ],
                "view": "timeSeries",
                "stacked": false,
                "region": "${instanceRegion}",
                "title": "Missed Calls - Widget",
                "period": 300,
                "stat": "Average",
                "yAxis": {
                    "left": {
                        "label": "Calls",
                        "showUnits": false 
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm",
                            "value": 20
                        }
                    ]
                }
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 0,
            "x": 18,
            "type": "metric",
            "properties": {
                "view": "timeSeries",
                "stacked": false,
                "metrics": [
                    [ "AWS/Lambda", "Errors", {"period": 60, "stat": "Sum", "color": "#d62728"} ]
                ],
                "region": "${instanceRegion}",
                "yAxis": {
                    "left": {
                        "label": "Errors",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 1
                        }
                    ]
                },
                "title": "Lambda Errors - Widget - Overall",
                "period": 300
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 0,
            "x": 0,
            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/Lambda", "Duration", {"region": "${instanceRegion}", "color": "#ff9896", "label": "Duration"} ]
                ],
                "view": "timeSeries",
                "stacked": false,
                "region": "${instanceRegion}",
                "period": 60,
                "yAxis": {
                    "left": {
                        "label": "Milliseconds",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "stat": "Sum",
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold (15 Minutes)",
                            "value": 900000
                        }
                    ]
                },
                "title": "Lambda Execution Duration - Widget"
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 6,
            "x": 6,
            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/Connect", "CallsPerInterval", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "period": 3600, "stat": "Sum", "color": "#2ca02c" } ]
                ],
                "period": 300,
                "stat": "Sum",
                "yAxis": {
                    "left": {
                        "label": "Calls",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                }
                ,
                "region": "${instanceRegion}",
                "title": "Total Calls - Widget",
                "liveData": true,
                "legend": {
                    "position": "bottom"
                }
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 6,
            "x": 12,
            "type": "metric",
            "properties": {
                "view": "timeSeries",
                "metrics": [
                    [ { "expression": "SEARCH('{AWS/Connect,ContactFlowName,InstanceId,MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "color": "#d62728" } ]
                ],
                "region": "${instanceRegion}",
                "period": 300,
                "yAxis": {
                    "left": {
                        "label": "Errors",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "title": "Contact Flow Errors - Widget - Overall"
            }
        },
        {
            "height": 6,
            "width": 6,
            "y": 6,
            "x": 18,
            "type": "metric",
            "properties": {
                "view": "timeSeries",
                "stacked": false,
                "metrics": [
                    [ { "expression": "SEARCH('{AWS/Connect,QueueName, InstanceId, MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]
                ],
                "region": "${instanceRegion}",
                "title": "Queue Wait Time - Widget - By Queue",
                "period": 300,
                "yAxis": {
                    "left": {
                        "label": "Seconds in Queue",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 600,
                            "fill": "above"
                        }
                    ]
                }
            }
        },
        {
            "height": 5,
            "width": 24,
            "y": 12,
            "x": 0,
            "type": "metric",
            "properties": {
                "view": "singleValue",
                "stacked": false,
                "metrics": [
                    [ { "expression": "SEARCH('{AWS/Connect,QueueName, InstanceId, MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]
                ],
                "region": "${instanceRegion}",
                "title": "Queue Wait Time - Widget - By Queue",
                "period": 300,
                "yAxis": {
                    "left": {
                        "label": "Seconds In Queue",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 600,
                            "fill": "above"
                        }
                    ]
                }
            }
        },
        {
            "height": 5,
            "width": 24,
            "type": "metric",
            "properties": {
                "view": "singleValue",
                "stacked": false,
                "metrics": [
                    [ { "expression": "SEARCH('{AWS/Lambda,FunctionName, InstanceId, MetricName}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]
                ],
                "region": "${instanceRegion}",
                "title": "Lambda Errors - Widget - By Lambda",
                "period": 300,
                "yAxis": {
                    "left": {
                        "label": "Errors",
                        "showUnits": false
                    },
                    "right": {
                        "label": "Time",
                        "showUnits": false
                    }
                },
                "annotations": {
                    "horizontal": [
                        {
                            "label": "Alarm Threshold",
                            "value": 600,
                            "fill": "above"
                        }
                    ]
                }
            }
        }
    ]
}

Solution

  • I was able to fix this. Here is the answer in case anyone else is looking.

    {
        "start": "-PT3H",
        "periodOverride": "inherit",
        "widgets": [
            {
                "height": 6,
                "width": 6,
                "y": 6,
                "x": 0,
                "type": "metric",
                "properties": {
                    "metrics": [
                        [ "AWS/Connect", "CallsPerInterval", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "visible": false } ],
                        [ ".", "ConcurrentCallsPercentage", ".", ".", ".", ".", { "visible": false } ],
                        [ ".", "ConcurrentCalls", ".", ".", ".", "." ],
                        [ ".", "MissedCalls", ".", ".", ".", ".", { "visible": false } ],
                        [ ".", "CallRecordingUploadError", ".", ".", ".", "CallRecordings", { "visible": false } ]
                    ],
                    "view": "timeSeries",
                    "stacked": false,
                    "region": "${instanceRegion}",
                    "title": "Concurrent Calls - Widget",
                    "period": 300,
                    "yAxis": {
                        "left": {
                            "label": "miliseconds",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "stat": "Average",
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 120
                            }
                        ]
                    }
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 0,
                "x": 6,
                "type": "metric",
                "properties": {
                    "metrics": [
                        [ "AWS/Connect", "ToInstancePacketLossRate", "Participant", "Agent", "Type of Connection", "WebRTC", "Instance ID", "${instanceId}", "Stream Type", "Voice", { "period": 60, "stat": "Average", "color": "#2ca02c" } ],
                        [ "...", { "period": 60, "stat": "Maximum", "color": "#d62728" } ]
                    ],
                    "view": "timeSeries",
                    "stacked": false,
                    "region": "${instanceRegion}",
                    "title": "Instance Packet Loss - Widget",
                    "yAxis": {
                        "left": {
                            "label": "Packet",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "period": 300,
                    "liveData": true,
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 0.8
                            }
                        ]
                    }
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 0,
                "x": 12,
                "type": "metric",
                "properties": {
                    "metrics": [
                        [ "AWS/Connect", "ConcurrentCallsPercentage", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "visible": false } ],
                        [ ".", "ConcurrentCalls", ".", ".", ".", ".", { "visible": false } ],
                        [ ".", "CallsPerInterval", ".", ".", ".", ".", { "visible": false } ],
                        [ ".", "MissedCalls", ".", ".", ".", "." ],
                        [ ".", "CallRecordingUploadError", ".", ".", ".", "CallRecordings", { "visible": false } ]
                    ],
                    "view": "timeSeries",
                    "stacked": false,
                    "region": "${instanceRegion}",
                    "title": "Missed Calls - Widget",
                    "period": 300,
                    "stat": "Average",
                    "yAxis": {
                        "left": {
                            "label": "Calls",
                            "showUnits": false 
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm",
                                "value": 20
                            }
                        ]
                    }
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 0,
                "x": 18,
                "type": "metric",
                "properties": {
                    "view": "timeSeries",
                    "stacked": false,
                    "metrics": [
                        [ "AWS/Lambda", "Errors", {"period": 60, "stat": "Sum", "color": "#d62728"} ]
                    ],
                    "region": "${instanceRegion}",
                    "yAxis": {
                        "left": {
                            "label": "Errors",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 1
                            }
                        ]
                    },
                    "title": "Lambda Errors - Widget - Overall",
                    "period": 300
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 0,
                "x": 0,
                "type": "metric",
                "properties": {
                    "metrics": [
                        [ "AWS/Lambda", "Duration", {"region": "${instanceRegion}", "color": "#ff9896", "label": "Duration"} ]
                    ],
                    "view": "timeSeries",
                    "stacked": false,
                    "region": "${instanceRegion}",
                    "period": 60,
                    "yAxis": {
                        "left": {
                            "label": "Milliseconds",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "stat": "Sum",
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold (15 Minutes)",
                                "value": 900000
                            }
                        ]
                    },
                    "title": "Lambda Execution Duration - Widget"
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 6,
                "x": 6,
                "type": "metric",
                "properties": {
                    "metrics": [
                        [ "AWS/Connect", "CallsPerInterval", "InstanceId", "${instanceId}", "MetricGroup", "VoiceCalls", { "period": 3600, "stat": "Sum", "color": "#2ca02c" } ]
                    ],
                    "period": 300,
                    "stat": "Sum",
                    "yAxis": {
                        "left": {
                            "label": "Calls",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    }
                    ,
                    "region": "${instanceRegion}",
                    "title": "Total Calls - Widget",
                    "liveData": true,
                    "legend": {
                        "position": "bottom"
                    }
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 6,
                "x": 12,
                "type": "metric",
                "properties": {
                    "view": "timeSeries",
                    "metrics": [
                        [ { "expression": "SEARCH('{AWS/Connect,ContactFlowName,InstanceId,MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "color": "#d62728" } ]
                    ],
                    "region": "${instanceRegion}",
                    "period": 300,
                    "yAxis": {
                        "left": {
                            "label": "Errors",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "title": "Contact Flow Errors - Widget - Overall"
                }
            },
            {
                "height": 6,
                "width": 6,
                "y": 6,
                "x": 18,
                "type": "metric",
                "properties": {
                    "view": "timeSeries",
                    "stacked": false,
                    "metrics": [
                        [ { "expression": "SEARCH('{AWS/Connect,QueueName, InstanceId, MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]
                    ],
                    "region": "${instanceRegion}",
                    "title": "Queue Wait Time - Widget - By Queue",
                    "period": 300,
                    "yAxis": {
                        "left": {
                            "label": "Seconds in Queue",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 600,
                                "fill": "above"
                            }
                        ]
                    }
                }
            },
            {
                "height": 5,
                "width": 24,
                "y": 12,
                "x": 0,
                "type": "metric",
                "properties": {
                    "view": "singleValue",
                    "stacked": false,
                    "metrics": [
                        [ { "expression": "SEARCH('{AWS/Connect,QueueName, InstanceId, MetricGroup}', 'Average', 300)", "id": "e1", "period": 300, "region": "${instanceRegion}" } ]
                    ],
                    "region": "${instanceRegion}",
                    "title": "Queue Wait Time - Widget - By Queue",
                    "period": 300,
                    "yAxis": {
                        "left": {
                            "label": "Seconds In Queue",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 600,
                                "fill": "above"
                            }
                        ]
                    }
                }
            },
            {
                "height": 5,
                "width": 6,
                "type": "metric",
                "x": 0,
                "y": 17,
                "properties": {
                    "metrics": [
                        [ { "expression": "SELECT SUM(Errors) FROM SCHEMA(\"AWS/Lambda\", FunctionName) GROUP BY FunctionName", "label": "Lambda", "id": "q1", "region": "${instanceRegion}", "period": 60, "stat": "Sum" } ]
                    ],
                    "view": "timeSeries",
                    "stacked": false,
                    "region": "us-west-2",
                    "title": "Lambda Errors - Widget - By Lambda",
                    "period": 60,
                    "yAxis": {
                        "left": {
                            "label": "Errors",
                            "showUnits": false
                        },
                        "right": {
                            "label": "Time",
                            "showUnits": false
                        }
                    },
                    "annotations": {
                        "horizontal": [
                            {
                                "label": "Alarm Threshold",
                                "value": 600,
                                "fill": "above"
                            }
                        ]
                    },
                    "stat": "Sum"
                }
            }
        ]
    }