Search code examples
pythonpython-3.xfunctionnamespacespython-internals

How many local variables can a Python (CPython implementation) function possibly hold?


We already know that Function arguments used to have the limit of 255 explicitly passed arguments. However, this behaviour is changed now and since Python-3.7 there's no limit except sys.maxsize which is actually the limit of python's containers. But what about the local variables?

We basically cannot add local variables to a function in a dynamic manner and/or changing the locals() dictionary is not permitted directly so that one can even test this in a brute force way. But the problem is that even if you change the locals() using compile module or exec function it doesn't affect the function.__code__.co_varnames, hence, you cannot access the variables explicitly inside the function.

In [142]: def bar():
     ...:     exec('k=10')
     ...:     print(f"locals: {locals()}")
     ...:     print(k)
     ...:     g = 100
     ...:     
     ...:     

In [143]: bar()
locals: {'k': 10}
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-143-226d01f48125> in <module>()
----> 1 bar()

<ipython-input-142-69d0ec0a7b24> in bar()
      2     exec('k=10')
      3     print(f"locals: {locals()}")
----> 4     print(k)
      5     g = 100
      6 

NameError: name 'k' is not defined

In [144]: bar.__code__.co_varnames
Out[144]: ('g',)

This means that even if you use a for loop like:

for i in range(2**17):
    exec(f'var_{i} = {i}')

The locals() will be contain 2**17 variables but you cannot do something like print(var_100) inside the function.

We know that basically there is no need to dynamically add a variable to the function while you can use a dictionary or in other words a custom namespace. But what's the proper way to test the limit of the maximum number of local variables in a function?


Solution

  • 2^32. The LOAD_FAST op used for loading local variables only has a 1-byte or 2-byte oparg depending on the Python version, but this can and will be extended up to 4 bytes by one or more EXTENDED_ARG ops, allowing access to 2^32 local variables. You can see some of the helpers used for EXTENDED_ARG in Python/wordcode_helpers.h. (Note that the opcode documentation for EXTENDED_ARG in the dis docs hasn't yet been updated to reflect the new Python 3.6 wordcode structure.)