How to get all metrics data of a particular resource in GCP

Question

Is there a way to get all the metric data for a specific resource?

I was trying with ListTimeSeries() method using the following request body

req := &monitoringpb.ListTimeSeriesRequest{
        Name:   "projects/" + projectID,
        Filter: `resource.type = "gce_instance" AND resource.labels.instance_id="123456789"`,
        Interval: &monitoringpb.TimeInterval{
            StartTime: &timestamp.Timestamp{Seconds: startTime},
            EndTime:   &timestamp.Timestamp{Seconds: endTime},
        },
        View: monitoringpb.ListTimeSeriesRequest_HEADERS,
    } 
iter := client.ListTimeSeries(ctx, req)

But we get an error stating:

The provided filter matches more than one metric. TimeSeries data are limited to a single metric per request.

Tried client.ListTimeSeries(ctx, req) from cloud.google.com/go/monitoring/apiv3/v2/monitoringpb package

Is there any way to get all metric type data in golang rather than specifying a particular metric type in request body?

Answer 1

The ListTimeSeriesRequest can only fetch metric data for a single metric type at a time. With the filter you applied, there can be many metric types. For example, your instance with instance_id as "123456789" can have multiple metric types like bytes_used, cpu_utilization, etc. So, we must pass the metric type as a filter. Ex: filter = 'metric.type = "compute.googleapis.com/guest/disk/bytes_used"'

To fetch all the metrics data for a particular resource, first we need to list out the metric types for that resource. Then we can iterate through the list of metric types and make the ListTimeSeriesRequest.

Here I've used Python monitoring API client to show sample:

from google.cloud import monitoring_v3
from collections import defaultdict
import time
import pprint
    
client = monitoring_v3.MetricServiceClient()
project_name = '''your project name'''
map = defaultdict(list)
    
for descriptor in client.list_metric_descriptors(name=project_name):
    if len(list(descriptor.monitored_resource_types)) != 0:
        resource_name = list(descriptor.monitored_resource_types).pop()
    else:
        continue
    metric_type = descriptor.type
    map[resource_name].append(metric_type)

start = time.time()
secs = int(start)
nanos = int((start - secs) * 10 ** 9)
end = seconds - 1200  # for 20mins interval from now
interval = monitoring_v3.TimeInterval(
        {
            "end_time": {"seconds": secs, "nanos": nanos},
            "start_time": {"seconds": end, "nanos": nanos},
        }
    )
#  To get metric data for all resources
for resource, metric_list in map.items():
    for metric_type in metric_list:
        result = monitoring_client.list_time_series(
        name=project_name, filter=f'metric.type = "{metric_type}"', interval=interval,
        view=monitoring_v3.ListTimeSeriesRequest.TimeSeriesView.FULL, retry=None)
        for res in result:
            pprint.pprint(res)
#  To get metric data for any resource like "gce_instance"
for metric_type in map['gce_instance']:
    result = monitoring_client.list_time_series(
    name=project_name, filter=f'metric.type = "{metric_type}"', interval=interval,
    view=monitoring_v3.ListTimeSeriesRequest.TimeSeriesView.FULL, retry=None)
    for res in result:
        pprint.pprint(res)

Similarly, you can iterate through list of metric_types for any resource and make a ListTimeSeriesRequest to fetch metrics data.