I am searching for a way to change pandas timestamps to python datetime objects and I am failing. I used to_pydatetime().
Below is my code with comments:
import pandas
import datetime
import pytz
forced_UTC = pytz.timezone("Europe/London").localize(datetime.datetime(2019, 1, 30, 9, 5)).tzinfo
forced_BST = pytz.timezone("Europe/London").localize(datetime.datetime(2019, 4, 27, 9, 5)).tzinfo
#print (forced_UTC, forced_BST)
# Create dataframe and check the value of the first element in the column.
my_df = pandas.DataFrame({"my_column": ["2019-04-01 00:15:00", "2019-02-23 13:00:00", "2019-02-23 14:00:00"]})
first_date = my_df["my_column"].iloc[0]
print (first_date, type(first_date)) # 2019-04-01 00:15:00 <class 'str'>
# Change every element in the column from string to datetime object.
my_df["my_column"] = [datetime.datetime.strptime(element, "%Y-%m-%d %H:%M:%S").replace(tzinfo=forced_UTC) for element in my_df["my_column"]]
first_date = my_df["my_column"].iloc[0]
print (first_date, type(first_date)) # <class 'pandas._libs.tslibs.timestamps.Timestamp'>
# Didn't work, thought it would make them python datetime objects.
# Try creating a new list and replacing the column.
list_to_replace = [element.to_pydatetime() for element in my_df["my_column"]] # make all timestamps datetimes
print (list_to_replace[0],type(list_to_replace[0])) # 2019-04-01 01:15:00+01:00 <class 'datetime.datetime'>
my_df["my_column"] = [element for element in list_to_replace]
first_date = my_df["my_column"].iloc[0]
print (first_date, type(first_date))# 2019-04-01 01:15:00+01:00 <class pandas._libs.tslibs.timestamps.Timestamp'>
# Didn't work.
# Use to_pydatetime() doesn't work either
print (first_date.to_pydatetime(), type(first_date)) # 2019-04-01 01:15:00+01:00 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
# Using to_pydatetime() like this works!?!?!?!
first_date = my_df["my_column"].iloc[0].to_pydatetime()
print (first_date, type(first_date)) # 2019-04-01 01:15:00+01:00 <class 'datetime.datetime'>
# print (datetime.datetime.strftime(first_date, "%H"))
# print (datetime.datetime(2019, 4, 1, 0, 15, tzinfo=forced_BST) > first_date)
Can anyone see what am I doing wrong?
It works, but you were printing the type without modifying the type.
Here, you are printing indeed a panda date (first_date) and a panda type (type(first_date)).
first_date = my_df["my_column"].iloc[0]
print (first_date, type(first_date)) # 2019-04-01 01:15:00+01:00 <class pandas._libs.tslibs.timestamps.Timestamp'>
# Didn't work.
In this case, you are printing a python date (first_date.to_pydatetime()) since you are indeed using to_pydatetime, but the first_date variable was never modified, so it's type is still a panda date .
print (first_date.to_pydatetime(), type(first_date)) # 2019-04-01 01:15:00+01:00 <class 'pandas._libs.tslibs.timestamps.Timestamp'>
And here you are also overwriting the first_date variable with the python date, (first_date = first_date.to_pydatetime()) that is why you consider it is working only then. So now first_date will be a python date, and type(first_date) will, consequently, return what you expected
# Using to_pydatetime() like this works!?!?!?!
first_date = my_df["my_column"].iloc[0].to_pydatetime()
print (first_date, type(first_date)) # 2019-04-01 01:15:00+01:00 <class 'datetime.datetime'>