Search code examples
pythonpandasdataframetime-seriesdrop

drop any day with Nan value in time series with 1 Min frequency Python

I have a time series dataframe with 1 min frequency. I need to drop any day which has one or more nan values. For example in the following df, days 2012-10-15 and 2012-10-25 need to be dropped.

import pandas as pd
index=pd.date_range(start='2012-10-15', end='2012-10-25', freq='1Min')
df=pd.DataFrame(range(len(index)), index=index, columns=['Number'])
df.iloc[1]=np.nan
df.iloc[-2]=np.nan
print(df)
Solution

  • You can use isna to check for nan and groupby.transform() on the date extracted by df.index.normalize():

    mask = df['Number'].isna().groupby(df.index.normalize()).transform('any')

df[~mask]