Search code examples
pandasdataframereindex

pandas dataframe NaN when reindexing with period_range


I want the code to add new rows not present in the dataframe. I'm getting NaN values when reindex-ing with period_range. I get the correct period_range but NaNs instead of preserving available values for column 'A'. Below is shown the code example:

I guess the problem comes out because using PeriodIndex and DatetimeIndex objects.

                      A
2018-10-31 14:08:26 NaN
2018-10-31 14:08:27 NaN
2018-10-31 14:08:28 NaN
2018-10-31 14:08:29 NaN
2018-10-31 14:08:30 NaN


import pandas as pd

data=[['2018-10-31 14:08:26', 1],
      ['2018-10-31 14:08:28', 2],
      ['2018-10-31 14:08:30', 3]]

df = pd.DataFrame(data=data, columns=['time','A'])
df.time = pd.to_datetime(df.time)
ts = df.time
idx = pd.period_range(min(ts), max(ts),freq='s')
df = df.set_index('time',drop=True)
df = df.reindex( idx )

Solution

  • data = [['2018-10-31 14:08:26', 1],
            ['2018-10-31 14:08:28', 2],
            ['2018-10-31 14:08:30', 3]]
    
    df = pd.DataFrame(data=data, columns=['time','A'])
    df['time'] = pd.to_datetime(df['time'])
    df.set_index('time').resample('S').asfreq()
    

    Output

    >>> df
                            A
    time    
    2018-10-31 14:08:26     1.0
    2018-10-31 14:08:27     NaN
    2018-10-31 14:08:28     2.0
    2018-10-31 14:08:29     NaN
    2018-10-31 14:08:30     3.0