Convert a string to datetime object and timestamp

Question

I have converted a string to date time object and from datetime object i got timestamp and when i convert back timestamp to datetime, i get different value.

str-->datetime format--> timestamp-->datetime

from datetime import datetime
import pandas as pd
import pytz

today = datetime.today().strftime('%Y:%m:%d')

col = ['HKH 13:30']
df = pd.DataFrame(col, columns=['snaptime'])
df['time_zone'] = df['snaptime'].str.split(' ', expand=True)[0]
df['time'] = df['snaptime'].str.split(' ', expand=True)[1]
df['timeobj'] = today + ' ' + df['time']
df['timeobj'] = df['timeobj'].apply(lambda x:datetime.strptime(x, '%Y:%m:%d %H:%M'))
hkhzone = pytz.timezone('Asia/Hong_Kong')
utc = pytz.timezone('UTC')
df['timeobj']  = df['timeobj'].apply(lambda x:hkhzone.localize(x))
df['UTC']  = df['timeobj'].apply(lambda x:x.replace(tzinfo=utc))
df['UTCTimestamp']  = df['UTC'].apply(lambda x:x.timestamp())
df['HKHtimestamp']  = df['timeobj'].apply(lambda x:x.timestamp())
df['UTCTime']  = df['UTCTimestamp'].apply(lambda x:datetime.fromtimestamp(x))
df['HKHtime']  = df['HKHtimestamp'].apply(lambda x:datetime.fromtimestamp(x))

Output:starting value is 13:30 but i finally get 11:00

Answer 1

There are actually a few issues.

1. You are not actually converting the time to UTC from Asia/Hong_Kong when you replace the timezone in timeobj.

2. When you calculate the 'UTCTimestamp' and 'HKHtimestamp', they are actually the same, as both are derived from the POSIX timestamp which is timezone-agnostic (it's the number of seconds since 1970-01-01 00:00:00 UTC).

3. When you convert 'UTCTimestamp' and 'HKHtimestamp' back to datetime, Python treats these timestamps as being in the local timezone of the machine running the script.

Below is an example of making everything timezone aware and using the builtin ZoneInfo instead of pytz.

from datetime import datetime, timezone
import pandas as pd
from zoneinfo import ZoneInfo

today = datetime.now(tz=timezone.utc).strftime('%Y-%m-%d')

col = ['HKH 13:30']
df = pd.DataFrame(col, columns=['snaptime'])
df['time_zone'] = df['snaptime'].str.split(' ', expand=True)[0]
df['time'] = df['snaptime'].str.split(' ', expand=True)[1]
df['timeobj'] = today + ' ' + df['time']
df['timeobj'] = df['timeobj'].apply(lambda x:datetime.strptime(x, '%Y-%m-%d %H:%M'))
hkhzone = ZoneInfo('Asia/Hong_Kong')
df['timeobj']  = df['timeobj'].apply(lambda x:x.replace(tzinfo=hkhzone))
df['UTCTimeObj']  = df['timeobj'].apply(lambda x:x.astimezone(ZoneInfo('UTC')))  
df['UTCTimestamp']  = df['UTCTimeObj'].apply(lambda x:x.timestamp())  
df['HKHtimestamp']  = df['timeobj'].apply(lambda x:x.timestamp())  
df['UTCTime']  = df['UTCTimestamp'].apply(lambda x:datetime.fromtimestamp(x, ZoneInfo('UTC')))  
df['HKHtime']  = df['HKHtimestamp'].apply(lambda x:datetime.fromtimestamp(x, hkhzone))  

display(df)

Take note of the following lines

# This is now timezone aware
today = datetime.now(tz=timezone.utc).strftime('%Y-%m-%d')

hkhzone = ZoneInfo('Asia/Hong_Kong')

df['timeobj']  = df['timeobj'].apply(lambda x:x.replace(tzinfo=hkhzone))

# Convert time to UTC
df['UTCTimeObj']  = df['timeobj'].apply(lambda x:x.astimezone(ZoneInfo('UTC')))  

# Convert UTC timestamp to datetime in UTC
df['UTCTime']  = df['UTCTimestamp'].apply(lambda x:datetime.fromtimestamp(x, ZoneInfo('UTC')))  

# Convert HKH timestamp to datetime in HKH timezone
df['HKHtime']  = df['HKHtimestamp'].apply(lambda x:datetime.fromtimestamp(x, hkhzone))