Search code examples
regexgoogle-cloud-platformgoogle-cloud-stackdrivergoogle-cloud-logginggoogle-cloud-monitoring

Create Distribution Metric with Logs-Based Metrics and Regex


I'm trying to create a metric extracted from a string from some logs.

I'm creating a new Distribution Log Metric. I'm applying the filter to get the right logs. The filter is working as I can preview it.

Then I'm entering the Field name for my metric. No issue here as I can find it in the proposed values.

Now the field is returning me a string with some query parameters. I want to convert the value of one of these query parameters as a metric. I'm adding a Regular expression to do so.

My string looks like this: https://blabla.com/Name=Foo&Value=123&Version=14 and this is my regex Version=([^&]*)

As suggested I tried the following regex Version=([0-9]+). Output is the same.

but the metric isn't working. When opening the Metrics explorer I have the following message:

Only numeric metric data can be drawn as a line chart. the data provided cannot be drawn

Anything I'm missing here?

#edit1

I tried something simpler. I have a field which is returning only value. For example "348" if I'm looking at the log. I tried to build a Distribution Metric based on this field using the following regex ([0-9.]+) and I have the same issue. The output doesn't seems to be read as numeric data.

#edit2

Adding some illustrations

the log with the requestSize field

enter image description here

The setup of the log-based metric with the regex

enter image description here

and the output

enter image description here


Solution

  • As mentioned by @c69, you can plot metrics with distribution values by using a heat map. Heatmaps use color to represent the values in the distribution. With heatmaps, you can overline percentile lines and you can configure these charts to only display outliers.

    If you want to use line charts, you must convert the histogram into a numerical value. One way to perform this conversion is to plot a specific percentile of the distribution.

    The percentile values for distribution metrics are computed, and the algorithm depends on the bucket counts, the bucket widths, and the shape of the histogram:

    • The 50th, 95th, and 99th percentile values are always different. However, they might be showing different percentiles within the same bucket.

    • The percentiles aren't generated from measurements because these values aren't available.

    • The width of the bucket determines the maximum error between the computed percentile and the measurements.

    • The number of samples in a histogram is important. For example, if this number is fewer than 20, then the 95th and 99th percentiles are always in the same bucket.

    • For any distribution metric, you can use the Cloud Monitoring API to determine the bucket model used for that metric. Because this model is timestamped, a service can change the bucket model.

    Refer to the link for more information about charting distribution metrics.