When the Python call method gets extra first argument?

The following sample

import types
import pprint
class A:
    def __call__(self, *args):
        pprint.pprint('[A.__call__] self=%r, args=%r'
                      % (self, list(args)))
class B:
    pass
if __name__ == '__main__':
    a = A()
    print(callable(a))
    a(1, 2)
    b = B()
    b.meth = types.MethodType(a, b)
    b.meth(3, 4)

prints

True
'[A.__call__] self=<__main__.A object at 0xb7233c2c>, args=[1, 2]'
('[A.__call__] self=<__main__.A object at 0xb7233c2c>, args=[<__main__.B '
 'object at 0xb71687cc>, 3, 4]')

The number of the __call__ method arguments is changed in the b.meth(3, 4) example. Please explain the first one (__main__.B object...) and when Python does provide it?

Using Python 3.5.3 on Debian 9.9 Stretch

Solution

The important concept here is that a class function is a function that has 'self' bound to it as its first argument.

I'll demonstrate in a couple of examples. The following code will be identical for all examples:

import types

# Class with function
class A:
    def func(*args):
        print('A.func(%s)'%(', '.join([str(arg) for arg in args])))

# Callable function-style class
class A_callable:
    def __call__(*args):
        print('A_callable.__call__(%s)'%(', '.join([str(arg) for arg in args])))

# Empty class
class B():
    pass

# Function without class
def func(*args):
    print('func(%s)'%(', '.join([str(arg) for arg in args])))

Now let's consider a couple of examples:

>>> func(42)
func(42)

This one is obvious. It just calls the function func with argument 42.

The next ones are more interesting:

>>> A().func(42)
A.func(<__main__.A object at 0x7f1ed9ed2908>, 42)
>>> A_callable()(42)
A_callable.__call__(<__main__.A_callable object at 0x7f1ed9ed28d0>, 42)

You can see that the class object self is automatically given to the function as the first argument. It is important to note that the self argument is not added because the function is stored in an object, but because the function was constructed as part of the object, and therefore has the object bound to it.

To demonstrate:

>>> tmp = A().func
>>> tmp
<bound method A.func of <__main__.A object at 0x7f1ed9ed2978>>
>>> tmp(42)
A.func(<__main__.A object at 0x7f1ed9ed2978>, 42)

>>> tmp = A_callable().__call__
>>> tmp
<bound method A_callable.__call__ of <__main__.A_callable object at 0x7f1ed9ed2908>>
>>> tmp(42)
A_callable.__call__(<__main__.A_callable object at 0x7f1ed9ed2908>, 42)

The self argument does not get added because you write a. before it. It is part of the function object itself, storing it in a variable still keeps that binding.

You can also manually bind a class object to a function, like this:

>>> tmp = types.MethodType(func, B)
>>> tmp
<bound method func of <class '__main__.B'>>
>>> tmp(42)
func(<class '__main__.B'>, 42)

On the other hand, just assigning a function to a class does not bind self to the function. As previously mentioned, the argument does not get dynamically added when called, but statically when constructed:

>>> b = B()
>>> b.func = func
>>> b.func
<function func at 0x7f1edb58fe18>
>>> b.func(42)
func(42) # does NOT contain the `self` argument

That is why we need to explicitely bind self to the function if we want to add it to an object:

>>> b = B()
>>> b.func = types.MethodType(func, b)
>>> b.func
<bound method func of <__main__.B object at 0x7f1ed9ed2908>>
>>> b.func(42)
func(<__main__.B object at 0x7f1ed9ed2908>, 42)

The only thing left is to understand how binding works. If a method func has a parameter a bound to it, and gets called with *args, it will add a to the beginning of *args and then pass it to the function. The beginning is important here.

Now we know everything needed to understand your code:

>>> a = A_callable()
>>> b = B()
>>> b.func = types.MethodType(a, b)
>>> b.func
<bound method ? of <__main__.B object at 0x7f1ed97e9fd0>>
>>> b.func(42)
A_callable.__call__(<__main__.A_callable object at 0x7f1ed97fb2b0>, <__main__.B object at 0x7f1ed97e9fd0>, 42)

First of all, we can change the b.func to plain tmp because, as previously discussed, adding a function to an object does not change its type or functionality. Only binding self does.

Then, let's step through the code piece by piece:

>>> a = A_callable()
>>> b = B()

So far so good. We have an empty object b and a callable object a.

>>> tmp = types.MethodType(a,b)

This line is the crux. If you understand this, you will understand everything.

tmp is now the function a with b bound to it. That means, if we call tmp(42), it adds b to the beginning of its arguments. a will therefore receive b, 42. Then, because a is callable, it forwards its arguments to a.__call__.

That means, we are at the point where tmp(42) is equal to a.__call__(b, 42).

Because __call__ is a class function of A_callable, a is automatically bound to the __call__ function during the construction of a. Therefore before the arguments reach A_callable.__call__, a gets added to the beginning of the argument list, meaning the arguments are now a, b, 42.

Now we are at the point where tmp(42) equals A_callable.__call__(a, b, 42). This is exactly what you see:

>>> tmp = types.MethodType(a, b)
>>> tmp(42)
A_callable.__call__(<__main__.A_callable object at 0x7f1ed97fb2b0>, <__main__.B object at 0x7f1ed97e9fd0>, 42)
>>> A_callable.__call__(a, b, 42)
A_callable.__call__(<__main__.A_callable object at 0x7f1ed97fb2b0>, <__main__.B object at 0x7f1ed97e9fd0>, 42)

Now if you split your arguments into self, *args, you basically just take away the first argument and store it in self. Your first argument is a, so self will be a, and your other *args will be b, 42. Again, this is exactly what you see.

When the Python __call__ method gets extra first argument?

When the Python call method gets extra first argument?