Search code examples
pythonpython-3.6superpython-3.8

`super` in a `typing.NamedTuple` subclass fails in python 3.8


I have code which worked in Python 3.6 and fails in Python 3.8. It seems to boil down to calling super in subclass of typing.NamedTuple, as below:

<ipython-input-2-fea20b0178f3> in <module>
----> 1 class Test(typing.NamedTuple):
      2     a: int
      3     b: float
      4     def __repr__(self):
      5         return super(object, self).__repr__()

RuntimeError: __class__ not set defining 'Test' as <class '__main__.Test'>. Was __classcell__ propagated to type.__new__?
In [3]: class Test(typing.NamedTuple): 
   ...:     a: int 
   ...:     b: float 
   ...:     #def __repr__(self): 
   ...:     #    return super(object, self).__repr__() 
   ...:                                                                         

>>> # works

The purpose of this super(object, self).__repr__ call is to use the standard '<__main__.Test object at 0x7fa109953cf8>' __repr__ instead of printing out all the contents of the tuple elements (which would happen by default). There are some questions on super resulting in similar errors but they:

  1. Refer to the parameter-less version super()
  2. Fail already in Python 3.6 (it worked for me before 3.6 -> 3.8 upgrade)
  3. I fail to understand how to fix this anyway, given that it's not a custom metaclass I have control over but the stdlib-provided typing.NamedTuple.

My question is how can I fix this while maintaining backwards compatibility with Python 3.6 (otherwise I'd just use @dataclasses.dataclass instead of inheriting from typing.NamedTuple)?

A side question is how can this fail at definition time given that the offending super call is inside a method which is not even executed yet. For instance:

In [3]: class Test(typing.NamedTuple): 
   ...:     a: int 
   ...:     b: float 
   ...:     def __repr__(self): 
   ...:         return foo 

works (until we actually call the __repr__) even though foo is an undefined reference. Is super magical in that regard?


Solution

  • I was slightly wrong in the other question (which I just updated). Apparently, this behavior manifests in both cases of super. In hindsight, I should have tested this.

    What's happening here is the metaclass NamedTupleMeta indeed doesn't pass __classcell__ over to type.__new__ because it creates a namedtuple on the fly and returns that. Actually, in Python's 3.6 and 3.7 (where this is still a DeprecationWarning), the __classcell__ leaks into the class dictionary since it isn't removed by NamedTupleMeta.__new__.

    class Test(NamedTuple):
        a: int
        b: float
        def __repr__(self):
            return super().__repr__()
    
    # isn't removed by NamedTupleMeta
    Test.__classcell__
    <cell at 0x7f956562f618: type object at 0x5629b8a2a708>
    

    Using object.__repr__ directly as suggested by Azat does the trick.

    how can this fail at definition time

    The same way the following also fails:

    class Foo(metaclass=1): pass
    

    Many checks are performed while the class is being constructed. Among these, is checking if the metaclass has passed the __classcell__ over to type_new.