I want to analyse GCP logs in real time and make alerts from it. Because analysis depends on some aggregations and correlations (example: event A happend less then 10 min from the event B so there should be an alert), the Cloud Logging - Logs Analytics seems as the perfect solution for that. I would like to run scheduled Cloud Function (let say every 5 min) that runs SQL queries on the Log bucket and if there is a match it will send an alert.
The problem is I didn't find it possible when reading the Logging API docs. I found this section: https://cloud.google.com/logging/docs/reference/v2/rpc/google.logging.v2#opsanalyticsquery but it says that this query can be run only via UI.
I also tried making Log Sink to a Big Query, but logs in a BigQuery arrived with much greater delay then in a Log Bucket.
Is there any other way around to make Logs Analytics query from cloud function? Or maybe there are some other GCP products better suited for this case?
You can create a linked dataset to be able to query your logs from BigQuery.
Thanks to this, you can use BigQuery to query your data, even to join them with external data (referential or others)