Do comprehensions in python call a function in the background?

I defined the following factorial function using recursion :

def factorial(n):
    if n == 0:
        return n
    else:
        return n*factorial(n-1)

Then, I defined the following function to return the least value for which a RecursionError is raised when calling the factorial function. (I already imported the sys module for this)

def test_recursion(n):
    sys.setrecursionlimit(n)
    num = 0
    while True:
        try:
            x = factorial(num)
            num += 1
        except:
            return num

I'm aware that this function is a bit misleading as calling the test_recursion function itself takes up one slot in the call stack.

Now, if I create the list l1 as follows :

l1 = []
for i in [10,11,12]:
    l1.append(test_recursion(i))

l2 is supposed to be the equivalent list, made by using list comprehension

l2 = [test_recursion(i) for i in [10,11,12]]

Now, print(l1, l2, sep = "\n" gives the following output :

[7,8,9]
[6,7,8]

Why does this happen? This happens with dictionary and set comprehensions as well.

Is it that comprehensions call some function in the background, which takes up a slot in the stack? How does the internal implementation of comprehension explain this?

Solution

Yes. Under Python 3.10.13, with these functions:

def test1():
    l1 = []
    for i in [10,11,12]:
        l1.append(test_recursion(i))
    return l1

def test2():
    l2 = [test_recursion(i) for i in [10,11,12]]
    return l2

Using the disassembler on the first snippet (dis.dis(test1)):

 20           0 BUILD_LIST               0
              2 STORE_FAST               0 (l1)

 21           4 LOAD_CONST               1 ((10, 11, 12))
              6 GET_ITER
        >>    8 FOR_ITER                 9 (to 28)
             10 STORE_FAST               1 (i)

 22          12 LOAD_FAST                0 (l1)
             14 LOAD_METHOD              0 (append)
             16 LOAD_GLOBAL              1 (test_recursion)
             18 LOAD_FAST                1 (i)
             20 CALL_FUNCTION            1
             22 CALL_METHOD              1
             24 POP_TOP
             26 JUMP_ABSOLUTE            4 (to 8)

 23     >>   28 LOAD_FAST                0 (l1)
             30 RETURN_VALUE

And on the second snippet:

 26           0 LOAD_CONST               1 (<code object <listcomp> at 0x1048a0d40, file "/Users/amadan/tmp/a.py", line 26>)
              2 LOAD_CONST               2 ('test2.<locals>.<listcomp>')
              4 MAKE_FUNCTION            0
              6 LOAD_CONST               3 ((10, 11, 12))
              8 GET_ITER
             10 CALL_FUNCTION            1
             12 STORE_FAST               0 (l2)

 27          14 LOAD_FAST                0 (l2)
             16 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x1048a0d40, file "/Users/amadan/tmp/a.py", line 26>:
 26           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                 6 (to 18)
              6 STORE_FAST               1 (i)
              8 LOAD_GLOBAL              0 (test_recursion)
             10 LOAD_FAST                1 (i)
             12 CALL_FUNCTION            1
             14 LIST_APPEND              2
             16 JUMP_ABSOLUTE            2 (to 4)
        >>   18 RETURN_VALUE

So you can see that the comprehension is treated as a function. I believe its purpose is to isolate the variables created inside the comprehension in their own scope. For example:

x = "original value"
[x for x in ["new value"]]
print(x)     # original value

x = "original value"
for x in ["new value"]:
    pass
print(x)     # new value