Confused about how to do a bulk get of metrics from the cloudwatch api

Question

I need to do some monitoring for my various amazon ec2 instances, so my plan was originally to create a dashboard with a number widget, and choose the various things I am interested in such as "CPUUtilization", and then I can fetch that dashboard and have access to all those current metric values. However, when I make the call to get that dashboard, the body I receive is:

{"widgets"=>[{"type"=>"metric", "x"=>0, "y"=>0, "width"=>24, "height"=>6, "properties"=>{"view"=>"singleValue", "metrics"=>[["AWS/EC2", "CPUUtilization", "InstanceId", "i-0ea0081d91f6e4aff"], [".", "DiskWriteBytes", ".", "."], [".", "CPUCreditUsage", ".", "."], [".", "StatusCheckFailed", ".", "."], [".", "NetworkPacketsIn", ".", "."], [".", "NetworkOut", ".", "."], [".", "StatusCheckFailed_System", ".", "."], [".", "CPUCreditBalance", ".", "."], [".", "StatusCheckFailed_Instance", ".", "."], [".", "NetworkIn", ".", "."], [".", "NetworkPacketsOut", ".", "."], [".", "DiskReadBytes", ".", "."], [".", "DiskWriteOps", ".", "."], [".", "DiskReadOps", ".", "."]], "region"=>"us-east-1"}}]}

It does not contain any metric data, only the names of the metrics.. So this is not helpful.

The list_metrics method looks promising, except for two things...

1) It wants a single metric? Even though the docs describe the method "List the specified metrics."

The input is an object, not an array, and to use this method, I have to do:

client.list_metrics({
  namespace: "AWS/EC2",
  metric_name: "CPUUtilization",
  dimensions: [
    {
      name: "InstanceId", 
      value: "i-0fc0181c98e6e4a66"
    },
  ],
})

Which gives me this object back:

#<struct Aws::CloudWatch::Types::ListMetricsOutput metrics=[#<struct Aws::CloudWatch::Types::Metric namespace="AWS/EC2", metric_name="CPUUtilization", dimensions=[#<struct Aws::CloudWatch::Types::Dimension name="InstanceId", value="i-0fc0181c98e6e4a66">]>], next_token=nil>

Which leads to 2) it returns just the metric name, no actual values...

What am I doing wrong?

How can I simply ask for a whole bunch of metrics for various instance ids, and get a response back that contains the actual metric values for the metrics I requested?

Answer 1

In short, you can't get data for multiple metrics in a single request to GetMetricStatistics API. Your approach of listing the metrics first is correct.

You can use ListMetrics API and set only the namespace and the metric name (remove the dimension where you're setting the exact instance). That should give you metrics for all instances that ran in the last 2 weeks. Then you need to loop over them and call GetMetricStatistics for each.

The dashboards API you tried to use is intended for manipulating dashboards (create, delete, edit dashboards and stuff on the dashboards). Those APIs won't give you any data. However, when dashboards APIs were release, they also published a blog post on how to use them. In that post they use an example of how to use the APIs to create a dashboard with metrics from all running EC2 instances. Maybe that would be useful to you:

https://aws.amazon.com/blogs/aws/new-api-cloudformation-support-for-amazon-cloudwatch-dashboards/