Search code examples
pythonpandasdatetimetimestampconcatenation

How to concatenate all dataframe column into one column?


I have a dataframe that looks roughly like:

           2020-01-01   2020-01-02   2020-01-03   2020-01-05
00:00:00     11             47          54           10 
01:00:00     12             49          46           22
...
23:00:00     15             34          22           40

Expected output...

2020-01-01 00:00:00   11
2020-01-01 01:00:00   12
...
2020-01-01 23:00:00   12
2020-01-02 00:00:00   47
2020-01-02 01:00:00   49
...
2020-01-01 23:00:00   34
...

Solution

  • Use DataFrame.melt with convert index to column, then convert columns to datetimes and timedeltas and join together with DataFrame.pop for extract columns:

    df = df.reset_index().melt('index')
    df.index = pd.to_datetime(df.pop('variable')) + pd.to_timedelta(df.pop('index'))
    
    #alternative with join strings
    #df.index = pd.to_datetime(df.pop('variable') + ' ' + df.pop('index'))
    print (df)
                         value
    2020-01-01 00:00:00     11
    2020-01-01 01:00:00     12
    2020-01-01 23:00:00     15
    2020-01-02 00:00:00     47
    2020-01-02 01:00:00     49
    2020-01-02 23:00:00     34
    2020-01-03 00:00:00     54
    2020-01-03 01:00:00     46
    2020-01-03 23:00:00     22
    2020-01-05 00:00:00     10
    2020-01-05 01:00:00     22
    2020-01-05 23:00:00     40
    

    If pop is not used is necessary remove columns by DataFrame.drop:

    df = df.reset_index().melt('index')
    df.index = pd.to_datetime(df['variable']) + pd.to_timedelta(df['index'])
    df = df.drop(['index','variable'], axis=1)
    

    Another idea with DataFrame.unstack and join MultiIndex with map and convert to DatetimeIndex, output is Series:

    s = df.unstack()
    s.index = pd.to_datetime(s.index.map('{0[0]} {0[1]}'.format))
    print (s)
    2020-01-01 00:00:00    11
    2020-01-01 01:00:00    12
    2020-01-01 23:00:00    15
    2020-01-02 00:00:00    47
    2020-01-02 01:00:00    49
    2020-01-02 23:00:00    34
    2020-01-03 00:00:00    54
    2020-01-03 01:00:00    46
    2020-01-03 23:00:00    22
    2020-01-05 00:00:00    10
    2020-01-05 01:00:00    22
    2020-01-05 23:00:00    40
    dtype: int64