Search code examples
splunksplunk-query

Splunk search if message is x for more than 5 minutes


I have two specific messages in splunk data that I'm searching for per user.

  • on-screen
  • off-screen

Anyone know how I can search in splunk for a user that is message="off-screen" for more than 5 minutes with a query checking every 2 minutes ?

index="document" (message="off-screen")

My query will be ran every 2 minutes so I want to check for the event with message off-screen. Then next time around check if 5 minutes have elapsed since the on-screen message was fired and that no on-screen event was fired in that time period for that user. Is this possible ?


Solution

  • If you want to find off-screen messages that don't have an on-screen message within 5 minutes, then you can use a transaction. Let's say your raw data is:

    | makeresults count=10
    | streamstats count
    | eval _time=_time-(count*60)
    | eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
    | eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
    | where NOT isnull(user)
    | table _time user message
    

    That would look like this:

    _time user message
    2021-05-28 13:57:50 Alice on-screen
    2021-05-28 13:56:50 Bob on-screen
    2021-05-28 13:53:50 Alice off-screen
    2021-05-28 13:50:50 Bob off-screen
    2021-05-28 13:49:50 Alice on-screen
    2021-05-28 13:48:50 Bob on-screen

    You need a transaction that gathers the user's cooresponding on-screen and off-screen messages as long as they are within 5 minutes. But you need to keep the orphans where the off-screen message doesn't have a cooresponding on-screen message. Then you filter out the transactions that have both and you get just the orphans:

    message="off-screen" OR message="on-screen"
    | transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
    | where mvcount(message)<2
    | table _time user message
    

    That would produce this output:

    _time user message
    2021-05-28 13:50:50 Bob off-screen

    Here is a runnable example:

    | makeresults count=10
    | streamstats count
    | eval _time=_time-(count*60)
    | eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
    | eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
    | where NOT isnull(user)
    | table _time user message
    
    | search message="off-screen" OR message="on-screen"
    | transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
    | where mvcount(message)<2
    | table _time user message