Search code examples
pandastimestamputc

pandas time shift from utc to local


I am trying to convert utc time to local time. This is what I had before

df_combined_features['timestamp'][1:10]
2013-01-24   2013-01-24 11:00:00
2013-04-25   2013-04-25 10:00:00
2013-07-25   2013-07-25 10:00:00
2013-10-24   2013-10-24 10:00:00
2014-01-30   2014-01-30 11:00:00
2014-04-24   2014-04-24 10:00:00
2014-07-24   2014-07-24 10:00:00
2014-10-23   2014-10-23 10:00:00
2015-01-27   2015-01-27 11:00:00

This is what I did

df_combined_features['time_stamp'].tz_localize('US/Central')[1:10]
2013-01-24 00:00:00-06:00   2013-01-24 11:00:00
2013-04-25 00:00:00-05:00   2013-04-25 10:00:00
2013-07-25 00:00:00-05:00   2013-07-25 10:00:00
2013-10-24 00:00:00-05:00   2013-10-24 10:00:00
2014-01-30 00:00:00-06:00   2014-01-30 11:00:00
2014-04-24 00:00:00-05:00   2014-04-24 10:00:00
2014-07-24 00:00:00-05:00   2014-07-24 10:00:00
2014-10-23 00:00:00-05:00   2014-10-23 10:00:00
2015-01-27 00:00:00-06:00   2015-01-27 11:00:00

I think it did the right thing, but I dont understand the output format. In particular

1) Why do the converted cols appear as the new index?

2) I understand that -06:00 (in the last row) is an hour shift, so the time is 6:00 am, how do I retrieve that information, the exact local time?

Desired output, I want the exact time to be posted, including the offset from utc. local time utc time

    2013-01-24 05:00:00   2013-01-24 11:00:00
    2013-04-25 05:00:00   2013-04-25 10:00:00
    2013-07-25 05:00:00   2013-07-25 10:00:00
    2013-10-24 05:00:00   2013-10-24 10:00:00
    2014-01-30 05:00:00   2014-01-30 11:00:00
    2014-04-24 05:00:00   2014-04-24 10:00:00
    2014-07-24 05:00:00   2014-07-24 10:00:00
    2014-10-23 05:00:00   2014-10-23 10:00:00
    2015-01-27 05:00:00   2015-01-27 11:00:00

Solution

  • When you call tz.localize you localize the index, if you want to modify the column you need to call dt.localize also to add the timezone offset call dt.tz_convert('UTC'):

    In [125]:
    df['timestamp'].dt.tz_localize('utc').dt.tz_convert('US/Central')
    
    Out[125]:
    index
    2013-01-24   2013-01-24 05:00:00-06:00
    2013-04-25   2013-04-25 05:00:00-05:00
    2013-07-25   2013-07-25 05:00:00-05:00
    2013-10-24   2013-10-24 05:00:00-05:00
    2014-01-30   2014-01-30 05:00:00-06:00
    2014-04-24   2014-04-24 05:00:00-05:00
    2014-07-24   2014-07-24 05:00:00-05:00
    2014-10-23   2014-10-23 05:00:00-05:00
    2015-01-27   2015-01-27 05:00:00-06:00
    Name: timestamp, dtype: datetime64[ns, US/Central]
    

    Compare without .dt:

    In [126]:    
    df['timestamp'].tz_localize('utc').tz_convert('US/Central')
    Out[126]:
    index
    2013-01-23 18:00:00-06:00   2013-01-24 11:00:00
    2013-04-24 19:00:00-05:00   2013-04-25 10:00:00
    2013-07-24 19:00:00-05:00   2013-07-25 10:00:00
    2013-10-23 19:00:00-05:00   2013-10-24 10:00:00
    2014-01-29 18:00:00-06:00   2014-01-30 11:00:00
    2014-04-23 19:00:00-05:00   2014-04-24 10:00:00
    2014-07-23 19:00:00-05:00   2014-07-24 10:00:00
    2014-10-22 19:00:00-05:00   2014-10-23 10:00:00
    2015-01-26 18:00:00-06:00   2015-01-27 11:00:00
    Name: timestamp, dtype: datetime64[ns]