I have fixed a bug recently, but still don't know why it ocurred. I had the following convert a timestamp (epoch) to datetimefield in Django model:
our_timezone = pytz.timezone("Asia/Jerusalem")
# e13 is used for true division
final_dict["published_date"] = datetime.datetime.fromtimestamp(
float(article_as_dict["publish_date"]) / 1e3).replace(tzinfo=our_timezone)
The above returned a datetime object that was always 39 minutes off the true epoch time (in our timezone).
I managed to fix it by removing replace, and instead putting the timezone inside the fromtimestamp method:
final_dict["published_date"] = datetime.datetime.fromtimestamp(
float(article_as_dict["publish_date"]) / 1e3, tz=our_timezone)
So what's the difference between specifying the tz=our_timezone inside the fromtimestamp method, and doing .replace(tzinfo=our_timezone) after creating the object? why does the .replace return the wrong time?
>>> datetime.fromtimestamp(1000000000).replace(tzinfo=our_timezone)
datetime.datetime(2001, 9, 9, 3, 46, 40, tzinfo=<DstTzInfo 'Asia/Jerusalem' LMT+2:21:00 STD>)
>>> datetime.fromtimestamp(1000000000, tz=our_timezone)
datetime.datetime(2001, 9, 9, 4, 46, 40, tzinfo=<DstTzInfo 'Asia/Jerusalem' IDT+3:00:00 DST>)
Notice the odd timezone offset in the first case.
Timezones are a bundle of multiple offsets. A timezone offset changes over time. Not only does it vary within a year based on summer and winter time, it varied historically when timezone data was revised and updated. You can inspect the timezone object to get some interesting historical data, reaching back many decades:
>>> our_timezone._utc_transition_times
[datetime.datetime(1, 1, 1, 0, 0), datetime.datetime(1901, 12, 13, 20, 45, 52), datetime.datetime(1917, 12, 31, 21, 39, 20), datetime.datetime(1940, 5, 31, 22, 0), datetime.datetime(1942, 10, 31, 21, 0), datetime.datetime(1943, 4, 1, 0, 0), datetime.datetime(1943, 10, 31, 21, 0), datetime.datetime(1944, 3, 31, 22, 0), datetime.datetime(1944, 10, 31, 21, 0), datetime.datetime(1945, 4, 15, 22, 0), datetime.datetime(1945, 10, 31, 23, 0), datetime.datetime(1946, 4, 16, 0, 0), datetime.datetime(1946, 10, 31, 21, 0), datetime.datetime(1948, 5, 22, 22, 0), datetime.datetime(1948, 8, 31, 20, 0), datetime.datetime(1948, 10, 31, 23, 0), datetime.datetime(1949, 4, 30, 22, 0), datetime.datetime(1949, 10, 31, 23, 0), datetime.datetime(1950, 4, 15, 22, 0), datetime.datetime(1950, 9, 15, 0, 0), datetime.datetime(1951, 3, 31, 22, 0), datetime.datetime(1951, 11, 11, 0, 0), datetime.datetime(1952, 4, 20, 0, 0), datetime.datetime(1952, 10, 19, 0, 0), datetime.datetime(1953, 4, 12, 0, 0), datetime.datetime(1953, 9, 13, 0, 0), datetime.datetime(1954, 6, 12, 22, 0), datetime.datetime(1954, 9, 11, 21, 0), datetime.datetime(1955, 6, 11, 0, 0), datetime.datetime(1955, 9, 10, 21, 0), datetime.datetime(1956, 6, 2, 22, 0), datetime.datetime(1956, 9, 30, 0, 0), datetime.datetime(1957, 4, 29, 0, 0), datetime.datetime(1957, 9, 21, 21, 0), datetime.datetime(1974, 7, 6, 22, 0), datetime.datetime(1974, 10, 12, 21, 0), datetime.datetime(1975, 4, 19, 22, 0), datetime.datetime(1975, 8, 30, 21, 0), datetime.datetime(1985, 4, 13, 22, 0), datetime.datetime(1985, 9, 14, 21, 0), datetime.datetime(1986, 5, 17, 22, 0), datetime.datetime(1986, 9, 6, 21, 0), datetime.datetime(1987, 4, 14, 22, 0), datetime.datetime(1987, 9, 12, 21, 0), datetime.datetime(1988, 4, 9, 22, 0), datetime.datetime(1988, 9, 3, 21, 0), datetime.datetime(1989, 4, 29, 22, 0), datetime.datetime(1989, 9, 2, 21, 0), datetime.datetime(1990, 3, 24, 22, 0), datetime.datetime(1990, 8, 25, 21, 0), datetime.datetime(1991, 3, 23, 22, 0), datetime.datetime(1991, 8, 31, 21, 0), datetime.datetime(1992, 3, 28, 22, 0), datetime.datetime(1992, 9, 5, 21, 0), datetime.datetime(1993, 4, 1, 22, 0), datetime.datetime(1993, 9, 4, 21, 0), datetime.datetime(1994, 3, 31, 22, 0), datetime.datetime(1994, 8, 27, 21, 0), datetime.datetime(1995, 3, 30, 22, 0), datetime.datetime(1995, 9, 2, 21, 0), datetime.datetime(1996, 3, 14, 22, 0), datetime.datetime(1996, 9, 15, 21, 0), datetime.datetime(1997, 3, 20, 22, 0), datetime.datetime(1997, 9, 13, 21, 0), datetime.datetime(1998, 3, 19, 22, 0), datetime.datetime(1998, 9, 5, 21, 0), datetime.datetime(1999, 4, 2, 0, 0), datetime.datetime(1999, 9, 2, 23, 0), datetime.datetime(2000, 4, 14, 0, 0), datetime.datetime(2000, 10, 5, 22, 0), datetime.datetime(2001, 4, 8, 23, 0), datetime.datetime(2001, 9, 23, 22, 0), datetime.datetime(2002, 3, 28, 23, 0), datetime.datetime(2002, 10, 6, 22, 0), datetime.datetime(2003, 3, 27, 23, 0), datetime.datetime(2003, 10, 2, 22, 0), datetime.datetime(2004, 4, 6, 23, 0), datetime.datetime(2004, 9, 21, 22, 0), datetime.datetime(2005, 4, 1, 0, 0), datetime.datetime(2005, 10, 8, 23, 0), datetime.datetime(2006, 3, 31, 0, 0), datetime.datetime(2006, 9, 30, 23, 0), datetime.datetime(2007, 3, 30, 0, 0), datetime.datetime(2007, 9, 15, 23, 0), datetime.datetime(2008, 3, 28, 0, 0), datetime.datetime(2008, 10, 4, 23, 0), datetime.datetime(2009, 3, 27, 0, 0), datetime.datetime(2009, 9, 26, 23, 0), datetime.datetime(2010, 3, 26, 0, 0), datetime.datetime(2010, 9, 11, 23, 0), datetime.datetime(2011, 4, 1, 0, 0), datetime.datetime(2011, 10, 1, 23, 0), datetime.datetime(2012, 3, 30, 0, 0), datetime.datetime(2012, 9, 22, 23, 0), datetime.datetime(2013, 3, 29, 0, 0), datetime.datetime(2013, 10, 26, 23, 0), datetime.datetime(2014, 3, 28, 0, 0), datetime.datetime(2014, 10, 25, 23, 0), datetime.datetime(2015, 3, 27, 0, 0), datetime.datetime(2015, 10, 24, 23, 0), datetime.datetime(2016, 3, 25, 0, 0), datetime.datetime(2016, 10, 29, 23, 0), datetime.datetime(2017, 3, 24, 0, 0), datetime.datetime(2017, 10, 28, 23, 0), datetime.datetime(2018, 3, 23, 0, 0), datetime.datetime(2018, 10, 27, 23, 0), datetime.datetime(2019, 3, 29, 0, 0), datetime.datetime(2019, 10, 26, 23, 0), datetime.datetime(2020, 3, 27, 0, 0), datetime.datetime(2020, 10, 24, 23, 0), datetime.datetime(2021, 3, 26, 0, 0), datetime.datetime(2021, 10, 30, 23, 0), datetime.datetime(2022, 3, 25, 0, 0), datetime.datetime(2022, 10, 29, 23, 0), datetime.datetime(2023, 3, 24, 0, 0), datetime.datetime(2023, 10, 28, 23, 0), datetime.datetime(2024, 3, 29, 0, 0), datetime.datetime(2024, 10, 26, 23, 0), datetime.datetime(2025, 3, 28, 0, 0), datetime.datetime(2025, 10, 25, 23, 0), datetime.datetime(2026, 3, 27, 0, 0), datetime.datetime(2026, 10, 24, 23, 0), datetime.datetime(2027, 3, 26, 0, 0), datetime.datetime(2027, 10, 30, 23, 0), datetime.datetime(2028, 3, 24, 0, 0), datetime.datetime(2028, 10, 28, 23, 0), datetime.datetime(2029, 3, 23, 0, 0), datetime.datetime(2029, 10, 27, 23, 0), datetime.datetime(2030, 3, 29, 0, 0), datetime.datetime(2030, 10, 26, 23, 0), datetime.datetime(2031, 3, 28, 0, 0), datetime.datetime(2031, 10, 25, 23, 0), datetime.datetime(2032, 3, 26, 0, 0), datetime.datetime(2032, 10, 30, 23, 0), datetime.datetime(2033, 3, 25, 0, 0), datetime.datetime(2033, 10, 29, 23, 0), datetime.datetime(2034, 3, 24, 0, 0), datetime.datetime(2034, 10, 28, 23, 0), datetime.datetime(2035, 3, 23, 0, 0), datetime.datetime(2035, 10, 27, 23, 0), datetime.datetime(2036, 3, 28, 0, 0), datetime.datetime(2036, 10, 25, 23, 0), datetime.datetime(2037, 3, 27, 0, 0), datetime.datetime(2037, 10, 24, 23, 0)]
With datetime.fromtimestamp(1000000000) you create a naïve timestamp, one without timezone information. You then simply attach the timezone object to it with replace. That just causes Python to use the first of the many offset objects in that timezone object, which results in an odd offset from a hundred years ago.
However, supplying the timezone object as "context information" to fromtimestamp directly, that method can pick the correct offset applicable to the timestamp and correctly produces a coherent timestamp object.
You could also do this after the fact with astimezone:
>>> datetime.fromtimestamp(1000000000).astimezone(our_timezone)
datetime.datetime(2001, 9, 9, 4, 46, 40, tzinfo=<DstTzInfo 'Asia/Jerusalem' IDT+3:00:00 DST>)
This method also intelligently picks the applicable offset.
In a nutshell: replace == dumb, simply overwrite raw data, fromtimestamp, astimezone == smart, knows how to work with timezones.