Setting the datetime in a multiindex to last day of the month

Question

I have a Multiindex Dataframe and would like to change the date level, so that the date of the last value I have for each month is changed to the last day of the month. Any help is appreciated.

The DataFrame (rolling_cov) 123610 rows × 10 columns:

  Date                                 NoDur         Durbl           Manuf
  2018-12-27     NoDur                 0.000109      0.000112        0.000118
                 Durbl                 0.000112      0.000339        0.000238
                 Manuf                 0.000118      0.000238        0.000246
  2018-12-28     NoDur                 0.000109      0.000113        0.000117
                 Durbl                 0.000113      0.000339        0.000239
                 Manuf                 0.000117      0.000239        0.000242
  2018-12-31     NoDur                 0.000109      0.000113        0.000118
                 Durbl                 0.000113      0.000339        0.000239
                 Manuf                 0.000118      0.000239        0.000245

The Code I tried:

rolling_cov.index= 
rolling_cov.index.set_levels([rolling_cov.index.levels[0].
         apply(pd.to_datetime(df['Date'] , format="%Y%m") + MonthEnd(1))])

The error I received:

'DatetimeIndex' object has no attribute 'apply'

Answer 1

It may be easier to convert it to a Series first, change the values, and then replace the original index with the new one.

idx = df.index.levels[0]

ser = pd.Series(idx)
last_of_mon = ser.groupby(ser.dt.year * 100 + ser.dt.month).last()

ser = ser.apply(
    lambda x: x + pd.offsets.MonthBegin(1) - pd.offsets.Day(1)
        if x in last_of_mon.values
        else x
)

df.index.set_levels(ser, 0, inplace=True)

Notice that + pd.offsets.MonthBegin(1) - pd.offsets.Day(1) is used to change to the last of the month. If you use + pd.offsets.MonthEnd(1) on a date that is already the last of the month, it changes it to the last of the next month.