Search code examples
pythonintrospection

How to determine the method type of stdlib methods written in C


The classify_class_attrs function from the inspect module can be used to determine what kind of object each of a class's attributes is, including whether a function is an instance method, a class method, or a static method. Here is an example:

from inspect import classify_class_attrs

class Example(object):
    @classmethod
    def my_class_method(cls):
        pass

    @staticmethod
    def my_static_method():
        pass

    def my_instance_method(self):
        pass

print classify_class_attrs(Example)

This will output a list of Attribute objects for each attribute on Example, with metadata about the attribute. The relevant ones in these case are:

Attribute(name='my_class_method', kind='class method', defining_class=<class '__main__.Example'>, object=<classmethod object at 0x100535398>)
Attribute(name='my_instance_method', kind='method', defining_class=<class '__main__.Example'>, object=<unbound method Example.my_instance_method>)
Attribute(name='my_static_method', kind='static method', defining_class=<class '__main__.Example'>, object=<staticmethod object at 0x100535558>)

However, it seems that many objects in Python's standard library can't be introspected this way. I'm guessing this has something to do with the fact that many of them are implemented in C. For example, datetime.datetime.now is described with this Attribute object by inspect.classify_class_attrs:

Attribute(name='now', kind='method', defining_class=<type 'datetime.datetime'>, object=<method 'now' of 'datetime.datetime' objects>)

If we compare this to the metadata returned about the attributes on Example, you'd probably draw the conclusion that datetime.datetime.now is an instance method. But it actually behaves as a class method!

from datetime import datetime

print datetime.now()  # called from the class: 2014-09-12 16:13:33.890742
print datetime.now().now()  # called from a datetime instance: 2014-09-12 16:13:33.891161

Is there a reliable way to determine whether a method on a stdlib class is a static, class, or instance method?


Solution

  • I think you can get much of what you want, distinguishing five kinds, without relying on anything that isn't documented by inspect:

    • Python instance methods
    • Python class methods
    • Python static methods
    • Builtin instance methods
    • Builtin class methods or static methods

    But you can't distinguish those last two from each other with using CPython-specific implementation details.

    (As far as I know, only 3.x has any builtin static methods in the stdlib… but of course even in 2.x, someone could always define one in an extension module.)


    The details of what's available in inspect and even what it means are a little different in each version of Python, partly because things have changed between 2.x and 3.x, partly because inspect is basically a bunch of heuristics that have gradually improved over time.

    But at least for CPython 2.6 and 2.7 and 3.3-3.5, the simplest way to distinguish builtin instance methods from the other two types is isbuiltin on the method looked up from the class. For a static method or class method, this will be True; for an instance method, False. For example:

    >>> inspect.isbuiltin(str.maketrans)
    True
    >>> inspect.isbuiltin(datetime.datetime.now)
    True
    >>> inspect.isbuiltin(datetime.datetime.ctime)
    False
    

    Why does this work? Well, isbuiltin will:

    Return true if the object is a built-in function or a bound built-in method.

    When looked up on an instance, either a regular method or a classmethod-like method is bound. But when looked up on the class, a regular method is unbound, while a classmethod-like method is bound (to the class). And of course a staticmethod-like method ends up as a plain-old function when looked up either way. So, it's a bit indirect, but it will always be correct.*


    What about class methods vs. static methods?

    In CPython 3.x, builtin static and class method descriptors both return the exact same type when looked up on their class, and none of the documented attributes can be used to distinguish them either. And even if this weren't true, I think the way the reference is written, it's guaranteed that no functions in inspect would be able to distinguish them.

    What if we turn to the descriptors themselves? Yes, there are ways we can distinguish them… but I don't think it's something guaranteed by the language:

    >>> callable(str.__dict__['maketrans'])
    False
    >>> callable(datetime.datetime.__dict__['now'])
    True
    

    Why does this work? Well, static methods just use a staticmethod descriptor, exactly like in Python (but wrapping a builtin function instead of a function). But class and instance methods use a special descriptor type, instead of using classmethod wrapping a (builtin) function and the (builtin) function itself, as Python class and instance methods do. These special descriptor types, classmethod_descriptor and method_descriptor, are unbound (class and instance) methods, as well as being the descriptors that bind them. There are historical/implementation reasons for this to be true, but I don't think there's anything in the language reference that requires it to be true, or even implies it.

    And if you're willing to rely on implementation artifacts, isinstance(m, staticmethod) seems a lot simpler…

    All that being said, are there any implementations besides CPython that have both builtin staticmethods and classmethods? If not, remember that practicality beats purity…


    * What it's really testing for is whether the thing is callable without an extra argument, but that's basically the same thing as the documented "function or bound method"; either way, it's what you want.