Search code examples
graphitewhispergraphite-carbon

Graphite -- How to send data to carbon that have timestamps over a year old

I'm using graphite to store and plot data and I want to send data points to the carbon daemon that are over a year old (first data point is from Dec 12, 2013 and last is Jan 12, 2015) but whisper does not seem to be capable (at least, not with my current settings) of accepting values more than 24-hours old.

Whenever I send metrics over a year old to carbon (using the plain text protocol), it simply drops them. I can tell because when I run whisper-fetch.py on the *.wsp file it shows timestamps from the last 24-hours all of which hold no data. However, if I send the same data (using the same protocol) with current timestamps (or any timestamp within the last 24 hours) it retains the value and is present when I run whisper-fetch.py on the *.wsp file.

I don't have any aggregation rules setup because I'm only interested in raw data points and I have my retentions setup like so:

[pos]
pattern = ^pos\.
retentions = 1m:2y

I understand that this will store data points every minute for 2 years but it is not capable of accepting data points from the past.

Is there a proper retention that will store historical data which have timestamps prior to the last 24-hrs? Or is there a setting somewhere in the *.config files that allow importing of data prior to the last 24-hrs? Or is this "feature" a limitation of whisper/carbon?

UPDATE

I should mention that the metrics I'm sending are of the format:

"pos.amps (data here) (unix timestamp here)"

I've tried a few different retention policies, including using the legacy formatting: 

[pos]
pattern = ^pos\.
retentions = 120:500

Signifying to collect 1 data point every 120 seconds and store 500 data points but I've still had no luck. I read here (https://serverfault.com/questions/593157/graphite-shows-none-for-all-data-points-even-though-i-send-it-data) that whisper has some sort of query period that is set to a default of 24-hours. That is the value I would like to change but I don't know where to change it!

Solution

  • I've found my mistake, whisper-fetch.py has a default time range of the past 24 hours unless specified otherwise. One I changed the call to the format

    ./whisper-fetch.py --pretty --from=TIMESTAMP --until=TIMESTAMP /path/to/whisper/file.wsp

    Everything worked fine and my data was there.

    Another useful tip would be to use graphite's URL api to query the database and check for the data there. Here's some documentation: http://graphite.readthedocs.org/en/latest/render_api.html