Search code examples
pythonpython-internals

Implementing block scoping in Python


I am trying to implement something akin to block scoping in Python for fun. This is what I’ve arrived at:

import inspect


class Scope:
    def __init__(self, *args):
        self._start = inspect.currentframe().f_back.f_locals.copy()
        print(self._start)

    def __enter__(self, *args):
        pass

    def __exit__(self, *args):
        for name in inspect.currentframe().f_back.f_locals.copy():
            if name not in self._start:
                print(f"deleting {name}")
                # print(globals())
                # del globals()[name]
                del inspect.currentframe().f_back.f_locals[name] # this doesn't do anything
                print(inspect.currentframe().f_back.f_locals) # no change


def compute(foo):
    with Scope():
        bar = foo ** 2
        print(bar)
    return bar  # this should fail


compute(5)

Unfortunately it seems that the frames other than the current one cannot be modified. (probably for good reason? :P). I’ve also tried modifying the globals dict and that works, but only for the case where the created lexical scope is at the module level. I would like my solution to work inside functions, etc.

I’ve tried to use the with API to mimic scopes in other languages. I’d be happy to hear of a solution using a different technique that would potentially work.

TLDR: I want to have some kind of construct that automatically destroys all local variables bound inside it on exit.


Solution

  • The locals variable in a frame cannot be written - it can be read, though. The main reason for that is that although variables in any scope are representable as a dictionary, for efficiency at some point variables started being stored in a different way, in slots tied to the frame itself, but not visible from Python code. This way the operations for reading and writing a variable are not indirected through a dictionary, and there is a simple linear index for local variables that is used.

    Whenever a call to locals() or the .f_locals member of a frame is accessed, cPython transparently copies the current values in the local variables to the resulting dictionary.

    Furthermore, the other storage of local variable,s called "fast storage" implies that space is reserved for all local variables at compile time fr a code object - and functions are compiled to a single code object. That means that even if a variable is used only inside a block, it will exist function-wide.

    This has been the behavior of Python for a long time, but it was somewhat recently documented in PEP 558: https://www.python.org/dev/peps/pep-0558/

    For global variables there is no such mechanism, and affecting the dictionary returned by globals or frame.f_globals will change the corresponding variables - that is why your idea works for global vars.

    The closest mechanism there is to that in the language is at the end of except blocks - at that point Python will delete the exception variable (to avoid having references to objects in the exception itself, which typically won't be reused after the except block.

    If you want something that is actually useful, not just as a toy, one thing I've used 2 or 3 times is to have a context object that can be changed within a with block, and will restore its values upon exit - so that previous values are restored. This object is usually a "context" thing - for example, with parameters like foreground and linewidth for a drawing API, or number of decimal places and rounding strategies for Decimal numbers.

    from collections import ChainMap
    
    class Context:
        def __init__(self):
            self._stack = [{}]
            
        def __enter__(self):
            self._stack.append({})
        
        def __exit__(self, *args, **kw):
            self._stack.pop()
            
        def __getattr__(self, name):
            if name.startswith("_"):
                return super().__getattr__(self, name)
            return ChainMap(*reversed(self._stack))[name]
        
        def __setattr__(self, name, value):
            if name.startswith("_"):
                return super().__setattr__(name, value)
            self._stack[-1][name] = value
            
        def __delattr__(self, name):
            if name.startswith("_"):
                return super().__deltattr__(name)
            del self._stack[1][name]
    

    And here is this code working in the an ipython console:

    
    In [137]: ctx = Context() 
    
    In [138]: ctx.color = "red"                          
    
    In [139]: ctx.color       
    Out[139]: 'red'
    
    In [140]: with ctx: 
         ...:     ctx.color = "blue" 
         ...:     print(ctx.color) 
         ...:                 
    blue
    
    In [141]: ctx.color       
    Out[141]: 'red'
    

    (for a real good "production quality" Context class, you'd have to take in account thead safety and asyncio task-safety - but the above is the basic idea)