Search code examples
pythonpandasdataframedatetimepandas-resample

Panda time series resample + adjusting values linearly


Using python and pandas, how do I resample a time series to even 5-min intervals (offset=zero min from whole hours) while also adjusting the values linearly?

Hence, I want to turn this:

         value
00:01    2
00:05    10
00:11    22
00:14    28

into this:

         value
00:00    0
00:05    10
00:10    20
00:15    30

Please note how the "value"-column was adjusted.

  • For simplicity, I have chosen the values to be exactly 2 * number of minutes.
  • In real life, however, the values are not that perfect. Sometimes there will exist more than one value between two even 5-min intervals and sometimes more than one 5-min interval between two "real" values, so when resampling I need to, for each even 5-min interval, find the "real" values before and after that even 5-min interval, and calculate a linearly interpolated value from them.

PS.

There is a lot of information about this everywhere on the internet, but I still wasn't able to find a function (sum, max, mean, etc, or write my own functino) that could accompish what I wanted to do.


Solution

  • I have reconsidered the code because the requirement was omitted from the comments. Create a new data frame by combining the original data frame with a data frame that is extended to one minute. I linearly interpolated the new data frame and extracted the results in 5-minute increments. This is my understanding of the process. If I'm wrong, please give me another answer.

    import pandas as pd
    import numpy as np
    import io
    
    data = '''
    time value
    00:01 2
    00:05 10
    00:11 22
    00:14 28
    00:18 39
    '''
    df = pd.read_csv(io.StringIO(data), sep='\s+')
    df['time'] = pd.to_datetime(df['time'], format='%H:%M')
    time_rng = pd.date_range(df['time'][0], df['time'][4], freq='1min')
    df2 = pd.DataFrame({'time':time_rng})
    df2 = df2.merge(df, on='time', how='outer')
    df2 = df2.set_index('time').interpolate('time')
    df2.asfreq('5min')
        value
    time    
    1900-01-01 00:01:00 2.0
    1900-01-01 00:06:00 12.0
    1900-01-01 00:11:00 22.0
    1900-01-01 00:16:00 33.5