Search code examples
alertinfluxdbkapacitor

How to add a deadman's switch to an existing alert?


Say I have the following alert in tick.

stream
    |from()
        .measurement('cpu')
        .groupBy(*)
    |alert()
        .crit(lambda: "usage_idle" < 10)
        .topic('cpu')

I also want to be alerted if the cpu time series disappears. Can I add a |deadman(...) section after |alert() ?


Solution

  • I have not used the Deadman node before.

    Looking at the Alert documentation you should be able to call Deadman after it as deadman is a chaining method of alert.

    Quote:

    AlertNode

    Chaining Methods

    • Alert

    • ...

    • Deadman

    So the following should work;

     stream
        |from()
            .measurement('cpu')
            .groupBy(*)
        |alert()
            .crit(lambda: "usage_idle" < 10)
            .topic('cpu')
        |deadman(threshold, interval)
             ...
    

    However if it doesn't you should be able to store the output of stream node into a variable and then apply it to both alert and deadman nodes separately.

    Example:

     var data =  
       stream
         |from()
           .measurement('cpu')
           .groupBy(*)
    data     
      |alert()
        .crit(lambda: "usage_idle" < 10)
        .topic('cpu')
    
    data   
      |deadman(threshold, interval)
         ...
    

    References:

    AlertNode - https://docs.influxdata.com/kapacitor/v1.3/nodes/alert_node/#deadman