Create a dashboard graph with data points in logs

Question

I have a cloud-run job witch emits logs (json formated) with metrics I would like to plot in a graph for visualizations.

I have been able to grep the logs and display them in text on the dashboard but can't figure out how to parse it into something which would make a nice curve.

With log based metrics I have been able to get the number of times a log occurs but not the value from inside the json.

This is another way to address the problem I have in Push Custom metrics in google cloud-run where a side car was suggested, but that would make my setup unnecessarily complex. I think I might have been asking the wrong questions

Answer 1

I was able to solve this with log analytics. Where I created a query using JSON_EXTRACT_SCALAR and CAST( ... as FLOAT64) functions

SELECT
  TIMESTAMP_SECONDS(UNIX_SECONDS(timestamp)) AS timestamp,
  CAST(JSON_EXTRACT_SCALAR(json_payload.fields, '$.balance') AS FLOAT64) AS balance,
  JSON_EXTRACT_SCALAR(json_payload.fields, '$.address') AS address
FROM
  `myproject-test-58cb.global._Default._Default`
WHERE
  json_payload.fields IS NOT NULL
  AND JSON_EXTRACT_SCALAR(json_payload.fields, '$.balance') IS NOT NULL 
ORDER BY
  timestamp ASC
LIMIT
  10000

In log analytics editor I used the save to dashboard button.

Using automatic interval and average to elimitnate the need for all data points to be synchronized.