Search code examples
javadropwizardmetricsgraphite

Dropwizard Metric aggregation issues on graphite

I am noticing strange spikes in data in Graphite, and I am using Dropwizard metrics to send the metrics to Graphite. I am specifically using the Metrics meter class to gather metrics. I have code in place which more or less does the following

public void recordMetric(){
 metricRegistry.meter("count.employee").mark();
 if (employee.getDesignation().equalsIgnoreCase("manager"){
    metricRegistry.meter("count.employee.manager").mark();
 }
}

The problem is when I view the "m1_rate" on Graphite, there are a lot of instances where the "count.employee.manager" metric has a higher rate than "count.employee" . How is this possible? This is throwing off my other metrics and casts doubts on the viability of the metric. Has anyone run into this problem? Any pointers on how this problem can be fixed? I suspect, this might be due to some quirks in Graphite, but not sure. Any help would be greatly appreciated. Thanks!

Solution

  • How do you calculate rate? if using nonNegativeDerivative() - please apply derivative first, and then sumSeries() - not vise versa. Please check http://www.jilles.net/perma/2013/08/22/how-to-do-graphite-derivatives-correctly/

    Also you need to set up correct aggregation (sum) for counters like described in http://obfuscurity.com/2012/05/A-Precautionary-Tale-for-Graphite-Users