Search code examples
pythonpython-modulesys

How are changes to `sys.modules` propagated?


Is sys.modules shared by the whole Python process, so that changes in sys.modules are seen in different modules, and exec statements (even with exec("",{},{}))? Or are there different copies for different contexts?


Solution

  • Is sys.modules shared by the whole Python process

    With caveats, yes. Every module that does import sys in a global scope will (unless, for some reason, it finds a different sys from the standard library module; or unless it overwrites the name sys with something else; or unless it dels the name; or...) have a global name sys which refers to the same module object for the sys module. Therefore, sys.modules will be the same dictionary in each case.

    so that changes in sys.modules are seen in different modules

    Yes, but such changes do not impact on any module's existing imports (unless you are using sys.modules to access and then modify the module object itself; but then, you could have just as easily done that using its existing global name). They only impact what happens if an import statement is executed after the change (and only if the import process is not altered to prevent or modify that behaviour). The dictionary (created internally by behind-the-scenes magic, and made available as sys.modules via more magic, when sys is imported) is just a cache. Changes like this won't matter even within the same module.

    Consider:

    import sys
    sys.modules['sys'] = 'hax' # does not have any impact on `sys`
    sys.modules # works just fine
    

    Or, a more complex example:

    >>> def hax():
    ...     import sys # this is only a local name
    ...     sys.modules['foo'] = 'hi mom'
    ...     global foo # causes the name to be put in the global namespace
    ...     import foo # causes the cached string to be loaded as the "module"
    ... 
    >>> hax() # "import" our fake foo module
    >>> foo # it is not a module at all, but just a string
    'hi mom'
    >>> sys # as expected, this is not defined
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'sys' is not defined
    >>> import sys # but if we import it,
    >>> sys.modules['foo'] # we get the same module object, and the same dict
    'hi mom'
    >>> # which still contains that string
    

    Notice that it is import foo that causes foo to become a global variable. We can't use foo simply by changing sys.modules['foo'], and altering the latter won't impact the former. If we continue from above:

    >>> sys.modules['foo'] = 'hi dad' # the change is not reflected...
    >>> foo
    'hi mom'
    >>> import foo # until we reload from the cache.
    >>> foo
    'hi dad'
    

    and exec statements (even with exec("",{},{}))?

    This is completely unrelated. exec doesn't care about the contents of sys.modules, unless perhaps you use it to build the dict that you pass for the globals argument.