Search code examples
awkgraphopentsdbmetricbosun

Percentage with used memory in Bosun


I've got a problem with doing a specific graph in Bosun. This graph should contain hosts with the highest memory usage in percentage, but I can't find any usable metric to do that. Of course I have os.mem.used just like os.mem.percent_free, but for me it isn't quite helpful. I thought of grabbing two series from query, just like for the alerting: total number of moemry and used one, divide used by total and multiply it by 100.

The problem seems to be that I can't divide series, so the last chance is to write my own metric by greping and awking free command in Linux.


Solution

  • If you really want a graph of percent_used instead of percent_free you should use 100-q("sum:os.mem.percent_free{host=$hosts}", "1h", ""). Then if you want to filter the series on the graph you should use the filter function. Example you can use on the expression page:

    $hosts=ny-redis*|ny-devredis*
    $limit=5
    $avgfree=avg(q("sum:os.mem.percent_free{host=$hosts}", "1h", ""))
    $lowest_free=limit(sort($avgfree,"asc"),$limit)
    $percent_used=100-q("sum:os.mem.percent_free{host=$hosts}", "1h", "")
    filter($percent_used,$lowest_free)
    

    We have two dev instances and two prod instances, so with a limit of 5 I see all of them in the graph:

    bosun percent_used

    But you can change the expression to use $limit=2 and it would only show the top two:

    bosun percent_used top 2

    however note the scale has been truncated. Right now I don't think there is any way to specify what scale to use on the expression page or in an alert template, but you could grab the raw data and graph it using a different graphing library.