GCP logging payload from payload (Stackdrive)

Question

I have the following logs:

My Goal is to create a custom metric that I can use on Stackdriver to collect the amount (count) of items, in this case (chocolate, chips and cookies), however, even when I can see a low of record on the Metric, they never show up on stackdriver.

I used the regular expressions and the build shows:

Which makes me think I'm setting it the right way but obviously I'm doing something wrong.

Any ideas?

Answer 1

Logs-to-metrics

You may want to create a Counter type not a Distribution.

Your filter is correct if the list shown in the log viewer matches the criteria by which logs will be counted. The step that you're missing is that you should use items as labels such that items values (e.g. cookies, chips etc.) become your label values. Additionally, the values become open-ended and, if you subsequently add candies, these will be automatically included.

See:

https://cloud.google.com/logging/docs/logs-based-metrics/counter-metrics https://cloud.google.com/logging/docs/logs-based-metrics/labels

NB Your regular expression (\d+) matches + (one or more) \d (digits).

Custom metrics

Another approach for you to consider is instrumenting your application to emit these metrics|measurements directly. OpenCensus is a general-purpose solution that works well with Stackdriver. It is straightforward (many languages are supported) to augment your code to e.g. count occurrences of chips, cookies etc.

https://cloud.google.com/monitoring/custom-metrics/open-census

This may be a more 'pure' solution to your problem if you're able to edit the application code. It also permits more flexibility than logs-to-metrics.

Like logs, OpenCensus also provides you flexibility in its support for monitoring solutions other than Stackdriver. Unlike logs, [OpenCensus] metrics represent richly structured data rather than text that must be parsed to become as useful.