Unexpected behaviour when testing equality with naive and tz-aware datetime instances

Question

The following was produced in Python 3.9.7.

I am well aware that the comparison between tz-aware and naive datetime instances is not allowed in Python and raises a TypeError. However, when testing for equality (with the == and != operators) this is actually not the case. In fact, the comparison always returns False:

import datetime
import pytz

t_tz_aware = datetime.datetime(2020, 5, 23, tzinfo=pytz.UTC)
t_naive = datetime.datetime(2020, 5, 23)

# Prints 'False'.
print(t_tz_aware == t_naive)

# Raises TypeError: can't compare offset-naive and offset-aware datetimes.
print(t_tz_aware < t_naive)

I checked the source code of the datetime library and the function for comparing datetime objects has a parameter called allow_mixed (which defaults to False):

def _cmp(self, other, allow_mixed=False)

When set to True, which is the case when comparing using the == operator, it is possible to compare tz-aware and naive datetime instances. Otherwise, it raises a TypeError:

# When testing for equality, set allow_mixed to True.
# For all the other operators, it remains False.
def __eq__(self, other):
   if isinstance(other, datetime):
      return self._cmp(other, allow_mixed=True) == 0

if myoff is None or otoff is None:
   if allow_mixed:
      return 2 # arbitrary non-zero value
   else:
      raise TypeError("cannot compare naive and aware datetimes")

So, it really seems intended behaviour. In fact, Pandas' implementation of the comparisons of pandas.Timestamps and similar is consistent with this.

My question is, what is the reasoning? I suppose, like the name of the parameter says, this way we can filter collections of datetime objects that contain both naive and tz-aware instances (i.e., "mixed"). But wouldn't this just introduce a source of potential bugs and unintended behaviour? What am I missing?

EDIT after deceze's comments: this is in fact still "semantically correct" (i.e., the dates are for sure different).

Answer 1

As can be seen in the docs, up until Python 3.2 a TypeError was actually raised in these cases:

Changed in version 3.3: Equality comparisons between aware and naive datetime instances don’t raise TypeError.

In 2012 the Python developers considered the trade-off between these two issues:

• Raising a TypeError would make it easier to catch bugs caused by the grave mistake of mixing naive and aware datetime objects.
• In Python you can use the equality comparison on virtually any combination of objects. Raising a TypeError for datetime objects only would break that consistency.

Here's the pertinent discussion on the Python developer mailing list:

This is nice when your datetime objects are freshly created. It is not so nice when some of them already exist e.g. in a database (using an ORM layer). Mixing naive and aware datetimes is currently a catastrophe, since even basic operations such as equality comparison fail with a TypeError (it must be pretty much the only type in the stdlib with such poisonous behaviour).

Comparing aware and naive datetime objects doesn't make much sense but it's an easy mistake to make. I would say the TypeError is a sensible way to warn you while simply returning False could lead to much confusion.

You could say the same about equally "confusing" results, yet equality never raises TypeError (except between datetime instances):

>>> () == []
False

Raising an exception has very serious implications, such as making it impossible to put these objects in the same dictionary.

And even closer to home,

>>> date(2012,6,1) == datetime(2012,6,1)
`False`

I agree, equality comparison should not raise an exception.

Let's make it so.

-- --Guido van Rossum (python.org/~guido)

Looks like the arguments for removing the exception were stronger in this exchange. Guido van Rossum is the creator of the Python language and had the last word on issues like this. That's why he used to be called benevolent dicator for life. So after his "Let's make it so", the behaviour was changed so that naive and aware datetime object always compare unequal instead of raising a TypeError.