Search code examples
pythonpython-2.7python-internalspython-descriptors

function descriptor on builtin types like dict.fromkeys behaves different to a normal method


Why does the function descriptor for dict.fromkeys function differently then other normal functions.

First of all you can't access __get__ like this: dict.fromkeys.__get__ you have to get it from the __dict__. (dict.__dict__['fromkeys'].__get__)

And then it doesn't work like any other function because it will only let itself get bound to a dict.

This works fine what I expected:

class Thing:
    def __init__(self):
        self.v = 5

    def test(self):
        print self.v


class OtherThing:
    def __init__(self):
        self.v = 6

print Thing.test
Thing.test.__get__(OtherThing())

this however does something unexpected:

#unbound method fromkeys
func = dict.__dict__["fromkeys"]

but it's description differs from a normal unbound function to looks like: <method 'fromkeys' of 'dict' objects> instead of: <unbound method dict.fromkeys> which was what I

this works as expected:

func.__get__({})([1,2,3])

but you can't bind it to something else I understand It wouldn't work but that doesn't usually stop us:

func.__get__([])([1,2,3])

This fails with a type error from the function descriptor...:

descriptor 'fromkeys' for type 'dict' doesn't apply to type 'list'

Why does python differentiate builtin type functions and normal functions like this? And can we do this too? Can we make a function that will only be bound to the type it belongs to?


Solution

  • class_or_type.descriptor always calls descriptor.__get__(None, class_or_type), which is why you can't get __get__ from a built-in unbound method object. The (unbound) method type produced for __get__ on a Python 2 function explicitly implements __get__ because Python methods are much more flexible. So both builtin.descriptor and class.descriptor invoke .__get__(), but the object type that either returns differs, and for Python methods the object type proxies attribute access:

    Accessing attributes via __dict__, on the other hand, never calls __get__, so class_or_type.__dict__['fromkeys'] gives you the original descriptor object.

    Under the hood, types like dict, including their methods, are implemented in C code, not in Python code, and C is much more picky about types. You can't ever use functions that are designed to work with the internal implementation details of a dict object with a list instead, so why bother?

    That's really all there is to it; methods for built-in types are not the same thing as functions / methods implemented in Python, so while they mostly work the same, they can't work with dynamic types, and the implementation of how the descriptors work reflects that.

    And because Python functions (wrapped in methods or not), could work with any Python type (if so designed), unbound method objects support __get__ so that you can take any such object and stick it onto another class; by supporting __get__ they support being re-bound to the new class.

    If you want to achieve the same thing with Python types, you'll have to wrap the function objects in custom descriptors, then implement your own __get__ behaviour to restrict what is acceptable.


    Note that in Python 3, function.__get__(None, classobject) just returns the function object itself, not an unbound method object like Python 2 does. The whole unbound / bound method distinction, where unbound method objects restrict the type for the first argument when called, wasn't found to be all that useful so it was dropped.