Search code examples
graphitegraphite-carbon

Graphite data loss after first retention policy

I am inserting the data in Graphite db with below retention policy in storage-schemas.conf

[default_1min_for_1day]
pattern = .*
retentions = 10s:2m,20s:4m

I have inserted data for the metrics key and the data is lost after 2min , i am not able to get the data with the below render api , its lost after 2mins reached , below not able to fecth for past 3min or 1h or with currentdate

GET : http://localhost:50000//render?target=metrics.*.api.proxy.north.*.*.danna.*.success.*&format=json&noNullPoints=true&from=20200110
Solution

  • You can follow the github link: https://github.com/graphite-project/whisper/issues/289

    aggregationMethod will be applied on this retention policy when switching boundaries.

    First retention - 10s:5m means Graphite will store 30 datapoints (every 10 seconds for last 5 minutes) in archive 0.

    Please note, that it will always store these datapoints, even if no data arrived. In that case Graphite will put NULLs there.

    Then next retention - 1m:1d means that every minute whisper will take 6 of these 10s datapoints from archive 0, apply average() function and store them in archive 1. But please note that Whisper will do so only if at least 3 (number of datapoints - 6 multiplied by xFilesFactor = 0.5) or more points in archive 0 have values (i.e. not NULLs). Otherwise Whisper decides that it has not enough data to propagate and put also NULL instead.

    Etc - third retention 1h:30d means that 60 of datapoints from archive 1 will be aggregated using average function and propagated to archive 2, but only if at least 30 of them have value, etc.