Why does this mypy, slots, and abstract class hack work?

Question

I've got a relatively big Python project and in an effort to minimise debugging time I'm trying to emulate a few aspects of a lower-level language. Specifically

1. Ability to type cast (Static Typing)
2. Prevent dynamic attribute addition to classes.

I've been using mypy to catch type casting errors and I've been defining __slots__ in my class instances to prevent dynamic addition.

At one point I need a List filled with two different children class (they have the same parent) that have slightly different attributes. mypy didn't like the fact that there were calls to attributes for a list item that weren't present in ALL the list items. But then making the parent object too general meant that dynamic addition of variables present in the other child wasn't prevented.

To fix this I debugged/brute-forced myself to the following code example which seems to work:

from abc import ABCMeta
from typing import List

class parentclass(metaclass=ABCMeta):
    __slots__:List[str] = []
    name: None

class withb(parentclass):
    __slots__ = ['b','name']
    def __init__(self):
        self.b: int = 0 
        self.name: str = "john"

class withf(parentclass):
    __slots__ = ['f','name']
    def __init__(self):
        self.name: str = 'harry'
        self.f: int = 123


bar = withb()
foo = withf()

ls: List[parentclass] = [bar, foo]

ls[0].f = 12 ## Needs to fail either in Python or mypy

for i in range(1):
    print(ls[i].name)
    print(ls[i].b) ## This should NOT fail in mypy

This works. But I'm not sure why. If I don't initialise the variables in the parent (i.e. only set them to None or int) then they don't seem to be carried into the children. However if I give them a placeholder value e.g. f:int = 0 in the parent then they make it into the children and my checks don't work again.

Can anyone explain this behaviour to an idiot like me? I'd like to know just so that I don't mess up implementing something and introduce even more errors!

As an aside: I did try List[Union[withb, withf]] but that didn't work either!

Answer 1

Setting a name to a value in the parent creates a class attribute. Even though the instances are limited by __slots__, the class itself can have non-slotted names, and when an instance lacks an attribute, its class is always checked for a class-level attribute (this is how you can call methods on instances at all).

Attempting to assign to a class attribute via an instance doesn't replace the class attribute though. instance.attr = someval will always try to create the attribute on the instance if it doesn't exist (shadowing the class attribute). When all classes in the hierarchy use __slots__ (without a __dict__ slot), this will fail (because the slot doesn't exist).

When you just for f: None, you've annotated the name f, but not actually created a class attribute; it's the assignment of a default that actually creates it. Of course, in your example, it makes no sense to assign a default in the parent class, because not all children have f or b attributes. If all children must have a name though, that should be part of the parent class, e.g.:

class parentclass(metaclass=ABCMeta):
    # Slot for common attribute on parent
    __slots__:List[str] = ['name']
    def __init__(self, name: str):
        # And initializer for parent sets it (annotation on argument covers attribute type)
        self.name = name

class withb(parentclass):
    # Slot for unique attributes on child
    __slots__ = ['b']
    def __init__(self):
        super().__init__("john")  # Parent attribute initialized with super call
        self.b: int = 0  # Child attribute set directly

class withf(parentclass):
    __slots__ = ['f']
    def __init__(self):
        super().__init__('harry')
        self.f: int = 123

If the goal is to dynamically choose whether to use f or b based on the type of the child class, mypy understands isinstance checks, so you can change the code using it to:

if isinstance(ls[0], withf):  # Added to ensure `ls[0]` is withf before using it
    ls[0].f = 12 ## Needs to fail either in Python or mypy

for x in ls:
    print(x.name)
    if isinstance(x, withb):  # Added to only print b for withb instances in ls
        print(x.b) ## This should NOT fail in mypy

In cases where isinstance isn't necessary (you know the type, because certain indices are guaranteed to be withf or withb), you can explicitly cast the type, but be aware that this throws away mypy's ability to check; lists are intended as a homogeneous data structure, and making position important (a la tuple, intended as a heterogeneous container) is misusing them.