Search code examples
pythondatetimepandaspytztimestamp-with-timezone

tz-aware datetime series producing UTC-based .date() output in pandas series apply(lambda) operation


I'm working with Unix timestamps in Python 2.7 / pandas 0.17.1, and need to produce series of local dates (in Eastern US time). Time-zone-aware stamps seem to work as expected (giving the local date) when calling .date() on individual data points, but when working them as series, they seem to revert to UTC dates. Is there a way to get the series-based date output to match the local dates that are produced from calling .date() individually on the objects? (Thanks!)

def stamp_converter(ts):
    ser = pd.to_datetime(ts, unit='s', utc=True)
    ind = pd.DatetimeIndex(ser).tz_localize('UTC').tz_convert('US/Eastern')
    return ind

test_stamps = pd.Series([1396670200, 1405733045, 1448248441])  
# e.g., 4/05 in GMT, 4/04 in Eastern

local_dates = pd.Series(stamp_converter(test_stamp))

print local_dates

# 0   2014-04-04 23:56:40-04:00
# 1   2014-07-18 21:24:05-04:00
# 2   2015-11-22 22:14:01-05:00
# dtype: datetime64[ns, US/Eastern]

print local_dates.apply(lambda x: x.date())

# 0    2014-04-05
# 1    2014-07-19
# 2    2015-11-23
# dtype: object

for i, x in local_dates.iteritems():
    print x.date()

# 2014-04-04
# 2014-07-18
# 2015-11-22

Solution

  • Oops. Found it.

    print local_dates.dt.date
    
    # 0    2014-04-04
    # 1    2014-07-18
    # 2    2015-11-22
    # dtype: object
    

    Nonetheless: it's puzzling that the behavior differs between individual .date() operations and .apply(lambda x: x.date()). I'd be interested if anyone can explain why this happens.