I'm trying to specifically return metrics between a certain timeslot. I'm trying to put an alert on a specific metric with different watermarks depending on being at night or during the day.
When I query the daytime metric it works fine: sum(increase(some_counter_total[5m])) and hour() >4 <20
This of course works as the bigger than 4 but smaller than 23 is a valid range.
However, this becomes more interesting when one wants to do this during the night time, for the reverse timeslot: sum(increase(some_counter_total[5m])) and hour() >20 <4
The first condition bigger then 20 already eliminates the following smaller then 4. The same is true when reversing the order.
I have tried some different combinations of using or, but I must be doing it wrong as I never got the querie to work properly. Ideally, I think I would be searching for a way to do a reverse query sum(increase(some_counter_total[5m])) and WHEN NOT hour() >4 <20.
Mind that Prometheus operates on data vectors. Try negation (and vs. unless) to exclude the hours you don't want.
Day time:
sum(increase(some_counter_total[5m])) and hour() >=9 <21
Night time:
sum(increase(some_counter_total[5m])) unless hour() >=9 <21
This is probably not the most efficient solution, but I couldn't find another way such as your straightforward suggestion or something like WHERE value IN (4, 5, 6, <...night time hours...>)).