Last period not output in Kapacitor recorded/replayed data

Question

I’m trying to aggregate data over the previous week hourly, and compute summary statistics like median, mean, etc.

I’m using something like this:

var weekly_median = batch
    |query('''SELECT median("duration") as week_median
              FROM "db"."default"."profiling_metrics"''')
        .period(1w)
        .every(1h)
        .groupBy(*)
        .align()
    |influxDBOut()
        .database('default')
        .measurement('summary_metrics')

The query works as expected, except that when recording and replaying data to test with

kapacitor record batch -task medians -past 30d
kapacitor replay -task medians -recording $rid -rec-time

the data is missing for the last period (1 week in this case). If I change the period to 1 day, all data is replayed except the final day’s worth.

Is this a problem with my tickscript, the way I’m recording data, or the way I’m replaying it?

Answer 1

I see, I need to do the aggregation in Kapacitor, not Influx. This seems to be a known issue, but finding documentation on it was tricky. https://github.com/influxdata/kapacitor/issues/1257 and https://github.com/influxdata/kapacitor/issues/1258 were helpful. The solution is to instead do something like:

var weekly_median = batch
    |query('''SELECT "duration"
              FROM "db"."default"."profiling_metrics"
              WHERE "result" =~ /passed/''')
        .period(1w)
        .every(1h)
        .groupBy(*)
        .align()
    |median('duration')
        .as('week_median')
    |influxDBOut()
        .database('default')
        .measurement('summary_metrics')