Design a cloud alert whenever any gcp service reaches prescribed quota usage
I want to design an alerting system, which will send alert to a prescribed channel, whenever any GCP service will exceed prescribed percent of quota limit, across all the metrics. I have configured alerts for Run and Function manually for all the metrics, but I am stuck at GCS, BQ and Logging,
Edit:
Here is the specific case:
My cloud function is required to read logs using Cloud Logging API. This function is generating quota limit exceed error for ReadRequestsPerMinutePerProject for Cloud Logging.
I also tried to find a metric in Resource type: Logging and to be safe I also tried to find it in Resource type: Cloud Function.
But there is no defined metric matching the description in either of those Resource types.
Having quota alerts for every GCP feature seems complicated but we can try to split it in pieces and analyse. I will focus on the ones that you mentioned in your question.
However - there are limits imposed on so many parameters (even considering buckets alone) that having alert for everything is an overkill.
For BigQuery is the same; there are limits to consider such as Concurrent rate limit for interactive queries, Cross-region federated querying or Daily destination table update limit.
You have to define what you want to monitor and the design a proper solution;
- trace a proper logs in Cloud Logging that hold the information about quota usage
- create a log based metric utilising mentioned logs
- create an alert utilising the metric
You can read more about quotas for:
Similar topic has been discussed here.
Complete list of API's (over 500 to be exact) that have quotas/limits you can find in the IAM & Admin > Quotas.
------- UPDATE -------
You can try to limit the number of checks performed by the function or ask for increased quota for Cloud Logging API.
Have in mind that checking performed by the function are counted towards the quota.