Grafana: Can't alert on a function metric?

Question

I am sending two regular metrics to Grafana through statsD and plotting them as two series (A and B). The metrics are "total_users" and "unfound_users".

I am then creating a third series (C) which uses this expression:

asPercent(movingAverage(#A, '5min'),movingAverage(#B, '5min'))

Then I am attempting to create an alert on C that triggers when the max() of C over a 5 minute period is over 2. Essentially this would mean "trigger an alert when a 5-minute average of unfound_users is more than 2% of a 5-minute-average of total_users."

You can see that my derived series (C) is getting plotted properly, but the alerts do not seem to work at all. I'm just getting back empty series in the test response.

We are using Grafana v.6.1.6.

Here is the setup of the series:

The alert configuration:

And the empty test response:

I'd appreciate some input! Thanks!

Answer 1

I figured out a workaround. Instead of using

asPercent(movingAverage(#A, '5min'),movingAverage(#B, '5min')) 

as my derived metric I needed to use the full paths in place of the reference ids.

This worked:

asPercent(movingAverage(stats.counters.stg.some.full.path.unfound_users.count, '5min'),
movingAverage(stats.counters.stg.some.full.path.total_users.count, '5min'))