Search code examples
pythonpycharmpython-3.6abstract-base-class

@staticmethod return value


In Python 3.6, I was trying to define a property in an AbstractBaseClass; my first try was something like this (later I discovered I can omit @staticmethod):

class AnAbstractClass(ABC):
    @property
    @staticmethod
    @abstractmethod
    def my_property():
        pass

However PyCharm shows me a warning on the property decorator: PyCharm warning

As far as I understand, the @staticmethod decorator does not return a callable but something different. (I suspect this is also causing mypy to raise an error on the return value type in my code, however I cannot reproduce the issue in a smaller code sample).

What's going on here?


Solution

  • To understand why you are getting the warning you are getting, you need to understand a bit about both decorators and descriptors.

    Decorators

    A decorator is a callable that replaces the thing it is decorating, and assigns it to the same name in the namespace. Normally, decorators are used for functions and classes to add some functionality, like type checking or threading or something, but really they can return anything at all.

    Since the output of a decorator does not have to be of the same type as the input, or perform any of the same processing, the order of decorators very much matters. Decorators are applied in order from the one closest to the function, to the one at the top of the list. In your case, abstractmethod, then staticmethod, then property.

    Descriptors

    Descriptors define a fairly complex protocol that allows for customization of objects using the binding behavior they provide. For your purposes, you need to know that functions are descriptors, and that placing them into a class object uses this. When you invoke any descriptor that is defined in a class on the instance of that class, the descriptor protocol binds the descriptor to the instance using the descriptor's __get__ method. The descriptor itself doesn't even have to be a callable, and neither is the return value of __get__, even though in most cases it is expected to be. For a function, __get__ returns a closure that automatically passes self as the first positional argument.

    For example, given a class A with a method def b(self, arg):, and an instance of that class called a, doing a.b(arg) turns into A.b.__get__(a, A)(arg). So while b is defined to have two positional arguments, it only needs one to be passed in explicitly when called on an instance. However, when you call b through the class, e.g., A.b(a, arg), it is just a normal function, and you need to pass in all the parameters manually, including self.

    Putting it all together

    abstractmethod, property and staticmethod are all decorators that return callable descriptor objects. However, the __get__ method of their decriptors works a bit differently than the __get__ of a normal function object.

    abstractmethod creates a fairly normal class method, but it interacts with the metaclass ABCMeta, so that you get all kinds of useful errors when you try to instantiate a class with abstract methods. It does not modify the input parameters expected by the result in any way. In fact, the documentation hints at the fact that all the side-effects may be related to the metaclass, and that the original input is just passed through. The only thing to really keep in mind here is

    When abstractmethod() is applied in combination with other method descriptors, it should be applied as the innermost decorator, as shown in the following usage examples: ...

    Your code seems to be following that injunction. In fact abstractmethod has nothing to do with your warning, but it seemed like a good idea to mention it here anyway.

    staticmethod returns a callable that bypasses normal binding behavior to create a method that does not care what class or instance it is invoked from. In particular, a method bound using staticmethod.__get__ will pass its arguments through to your function, instead of prepending self first (i.e., __get__ basically just returns your original function). You can imagine that this would be a problem for something that expects to receive a self parameter, like the setter of a property.

    Unlike abstractmethod and staticmethod, property creates a data descriptor. This means that it returns an object that has both a __get__ binding and a __set__ binding (and also a __del__ binding). A property's __get__ method works pretty much just like a normal function's __get__ method, but applied specifically to the getter function. property cares very much what instance it is invoked from because of course you want different instances to have different values of the attribute the property wraps.

    So what you have in your code is staticmethod followed by property. The first decorator returns a function that does not prepend self to its argument list when bound, while the second expects one that does. There is nothing stopping you from invoking the decorators, but the IDE warning is telling you that you are not going to succeed in calling the resulting object. If you try to access my_property on a concrete implementation of AnyAbstractClass, you will probably get a TypeError telling you that my_property does not accept any positional parameters, but one was given, because property.__get__ will prepend self to the argument list of the static method, which does not accept any arguments.

    Keep in mind that applying staticmethod to the result of property won't help you much either. A property instance is not callable at all. It operates entirely through its __get__, __set__ and __del__ methods, while staticmethod assumes that you pass in a callable.

    Solution

    As you have correctly discovered, staticmethod and property don't mix well. A property, by its very nature, should always be aware of the instance it operates on. The correct way to go about doing this is to add a self parameter and allow normal method binding to take place.

    Both property and staticmethod play well with abstractmethod (as long as abstractmethod is applied first), because it makes effectively no change to your original function. In fact, the docs of abstractmethod mention specifically that the getter, setter or deleter of a property abstract makes the whole property abstract.

    TL;DR

    staticmethod returns a descriptor that is callable, but whose __get__ method returns an unbound version of itself. property creates an uncallable descriptor whose __get__ method calls the getter of the property. Using the resulting property will attempt to pass self to a static method which doesn't accept it.