Search code examples
pandastime-seriesmulti-indexreindexdatetimeindex

Pandas: convert timeseries columns to multi-index with DatetimeIndex


I have a csv file/dataframe of a time series that looks like this:

IDX_A   IDX_B   1/1/20  1/2/20  1/3/20
A       1       A1_0    A1_1    A1_2
A       2       A2_0    A2_1    A2_2
B       3       B3_0    B3_1    B3_2
B       4       B4_0    B3_1    B3_2

I'd like to convert to a multi-index with the first level as a DatetimeIndex:

                        F1
Date    IDX_A   IDX_B
1/1/20  A       1       A1_0
                2       A2_0
        B       3       B3_0
                4       B4_0
1/2/20  A       1       A1_1
                2       A2_1
        B       3       B3_1
                4       B3_1
1/3/20  A       1       A1_2
                2       A2_2
        B       3       B3_2
                4       B3_2

I'd think this has been asked before but I can only find information about going in the other direction for a single index. I'll be appending additional feature columns and using in existing code so this is best format for me, especially considering a DatetimeIndex makes sense for a time series.


Solution

  • My approach:

    (df.set_index(['IDX_A','IDX_B'])
       .rename_axis(columns='Date')
       .stack()
       .reorder_levels((2,0,1))
       .sort_index()
       .to_frame(name='F1')
    )
    

    Or using melt:

    (df.melt(['IDX_A','IDX_B'], var_name='Date',value_name='F1')
       .set_index(['Date','IDX_A','IDX_B'])
    )
    

    Output:

                          F1
    Date   IDX_A IDX_B      
    1/1/20 A     1      A1_0
                 2      A2_0
           B     3      B3_0
                 4      B4_0
    1/2/20 A     1      A1_1
                 2      A2_1
           B     3      B3_1
                 4      B3_1
    1/3/20 A     1      A1_2
                 2      A2_2
           B     3      B3_2
                 4      B3_2