Search code examples
pythonpandasdatetimetimestamputc

convert time to UTC in pandas

I have multiple csv files, I've set DateTime as the index.

df6.set_index("gmtime", inplace=True)
#correct the underscores in old datetime format
df6.index = [" ".join( str(val).split("_")) for val in df6.index]
df6.index = pd.to_datetime(df6.index)

enter image description here

The time was put in GMT, but I think it's been saved as BST (British summertime) when I set the clock for raspberry pi. I want to shift the time one hour backwards. When I use

df6.tz_convert(pytz.timezone('utc'))

it gives me below error as it assumes that the time is correct.

Cannot convert tz-naive timestamps, use tz_localize to localize

How can I shift the time to one hour?

Solution

  • Given a column that contains date/time info as string, you would convert to datetime, localize to a time zone (here: Europe/London), then convert to UTC. You can do that before you set as index.

    Ex:

    import pandas as pd

dti = pd.to_datetime(["2021-09-01"]).tz_localize("Europe/London").tz_convert("UTC")

print(dti) # notice 1 hour shift:
# DatetimeIndex(['2021-08-31 23:00:00+00:00'], dtype='datetime64[ns, UTC]', freq=None)

    Note: setting a time zone means that DST is accounted for, i.e. here, during winter you'd have UTC+0 and during summer UTC+1.