Google Cloud Monitoring MQL: Time of Day

Question

Is it possible to use MQL to include time of day in the alert condition?

Basic example in their documentation includes:

  fetch gce_instance :: compute.googleapis.com/instance/cpu/usage_time
  | rate (5m)
  | condition lt(val(), 0.5's{CPU}/min')

I essentially want to do this:

  fetch gce_instance :: compute.googleapis.com/instance/cpu/usage_time
  | rate (5m)
  | condition lt(val(), 0.5's{CPU}/min') and ( gt(end.time, '06:00:00') and lt(end.time, '18:00:00') )

Obviously that isn't valid, but I'm struggling to put the syntax together for time related functions from their documentation - does anyone know if this is possible?

Answer 1

I also want this so I can silence alerts during scheduled nightly upgrades. I could not find a way to do it with MQL, but here are some workarounds that might help:

1. Create a snooze. You can select a period of time over which the alert will be silenced. Due to its limitations it only works if your quiet period is rare:
   • You can only create them with the console
   • There is no "recurring snooze," so if you want a daily quiet period then you have to create a snooze every day using the console
2. Create a cloud scheduler job to enable and disable the alert using the alerting API.

I am trying the API approach. The enable/disable calls can be made from my upgrade job, so I don't think I need the scheduler. Here are the steps I used to work this out:

1. List the monitoring policies to get the policy name. Note conditions have names too. You want the policy name, not the condition name:

   gcloud alpha monitoring policies list
   

2. Disable/enable the policy:

   gcloud alpha monitoring policies update projects/<project-name>/alertPolicies/<policy-id> --no-enabled
   # Do my upgrade...
   gcloud alpha monitoring policies update projects/<project-name>/alertPolicies/<policy-id> --enabled