Instance method aliases to builtin-functions in Python

Question

Trying to code as efficient as possible object-oriented implementation of priority queue in Python, I faced an interesting behavior. The following code works fine

from heapq import heappush


class PriorityQueue(list):
    __slots__ = ()

    def push(self, item):
        heappush(self, item)

However, I really didn’t want to write a wrapper method for calling heappush, as it incurs additional overhead for calling the function. I reasoned that since the heappush signature uses list as the first argument, while aliasing the push class attribute with the heappush function, the latter becomes a full-fledged class instance method. However, my assumption turned out to be false, and the following code gives an error.

from heapq import heappush


class PriorityQueue(list):
    __slots__ = ()
    push = heappush


PriorityQueue().push(0)
# TypeError: heappush expected 2 arguments, got 1

But going to cpython heapq source code, just copying heappush implementation into the scope and applying the same logic works fine.

from heapq import _siftdown


def heappush(heap, item):
    """Push item onto heap, maintaining the heap invariant."""
    heap.append(item)
    _siftdown(heap, 0, len(heap) - 1)


class PriorityQueue(list):
    __slots__ = ()
    push = heappush


pq = PriorityQueue()
pq.push(0)
pq.push(-1)
pq.push(3)
print(pq)
# [-1, 0, 3]

• The first question: Why does it happen? How does Python decide which function is appropriate for binding as an instance method and which is not?
• The second question: What is the difference between heappush in the cpython/Lib/heapq.py and the actual heappush from the heapq module? They are actually different since the following code gives an error

from dis import dis
from heapq import heappush


dis(heappush)
# TypeError: don't know how to disassemble builtin_function_or_method objects

• The third question: How can one force Python to bind native heappush as an instance method? Some metaclass magic?

Thank you!

Answer 1

What takes place is that Python offers pure Python implementations of a lot of its algorithms in the standard library even when it contains acceletated native code implementations of the same algorithms.

The heapq library is one of those - if you pick the file you link to, but close to the end, you will see the code snippet which looks if the native version is available, and overwrites the Python version, which has the code you copy and pasted - https://github.com/python/cpython/blob/76cd81d60310d65d01f9d7b48a8985d8ab89c8b4/Lib/heapq.py#L580

try:
    from _heapq import *
except ImportError:
    pass
...

The native version of heappush is loaded into the module, and there is no easy way of getting a reference to the original Python function, short of getting to the actual file source code.

Now, the point: why do native functions do not work as class methods? heappush's type is builtin_function_or_method, in constrast with function for pure Python functions - and one of the major diference is that the second object type features a __get__ method. This __get__ makes Python defined functions work as "descriptors": the __get__ method is called when one retrieves the attribute from an instance. For ordinary functions, this call records the self parameter and injects it when the actual function is called.

Thus, it is easy to write an "instancemethod" decorator that will make built-in functions work as Python functions and usable as methods. However, the overhead of creating a partial or a lambda function should surpass the overhead of the extra function call you are trying to eliminate - so you should get no speed gains from it, although it might still read as more elegant:

class instancemethod:
    def __init__(self, func):
        self.func = func
    def __get__(self, instance, owner):
        return lambda *args, **kwargs: self.func(instance, *args, **kwargs)

import heapq

class MyHeap(list):
    push = instancemethod(heapq.heappush)