Python cyclic import unexpected behavior

Question

I have discovered something unexpected when playing with cyclic imports. I have two files in the same directory:

a.py

import b
print("hello from a")

b.py

import a
print("hello from b")

Running either python3 a.py and python3 b.py does not result in a cyclic import related error. I know that the first imported module is imported under the name __main__, but I still do not understand this behavior. For example, running python3 a.py or python -m a produces the following output:

hi from a
hi from b
hi from a

Looking at the output of print(sys.modules.keys()), I can see that both modules are somehow already imported when checking it, even when importing the sys module as the first thing in one of the modules. I did not use sys.modules properly before answering my own question.

This does not happen if neither of the cyclic imported modules is the __main__ module. My Python version is Python 3.6.3 on Ubuntu 17.10. It still happens, but there is a visible error only if there is actually something you use from one of the cyclically imported modules.

See my own answer for clarifications.

Answer 1

The answer to my question

I have discovered the answer. I will try to sketch an explanation:

Executing python3 a.py imports the module in file a.py as __main__:

• import b in module __main__:

  • import a in module b -> Imports the module in file a.py as a

  • import b in module a -> Nothing happens, already imported that module

  • print('hello from a') in a.py (executing module a)

  • import a in module b finished

• print('hello from b') in b.py (executing module b)

• import b in module __main__ finished
• print('hello from a') in a.py(executing module __main__)

The problem is that there is no cyclic import error per se. A module is imported only once, and after that, other imports of the same module can be seen as no-ops.

This operation can be seen as adding a key to the sys.modules dictionary corresponding to the name of the imported module and then setting attributes on the module object associated with that key as it gets executed. So if the key is already present in the dictionary (on a second import of the same module), nothing happens on the second import. The already imported above means already present in the sys.modules dictionary. This reflects the procedural nature of Python (being originally implemented in C) and the fact that anything in Python is an object.

The lurking problem

In order to show the fact that the problem associated with cyclic imports is still present, let's add a function to module b and try to use it from module a.

a.py

import b

b.f()

b.py

import a

def f():
    print('hello from b.f()')

Executing now python a.py imports the module in file a.py as __main__:

• import b in module __main__:

  • import a in module b -> Imports the module in file a.py as a

  • import b in module a -> Nothing happens, already imported that module

  • b.f() -> AttributeError: module 'b' has no attribute 'f'

Note: The line b.f() can be further simplified to b.f and the error will still occur. This is because b.f() first accesses the attribute f of module object b, which happens to be a function object, and then tries to call it. I wanted to point out again the object oriented nature of Python.

The from ... import ... statement

It is interesting to mention that using the from ... import ... form gives another error, even though the reason is the same:

a.py

from b import f

f()

b.py

import a

def f():
    printf('hello from b.f()')

Executing python a.py imports the module in file a.py as __main__:

• from b import f in module __main__ actually imports the whole module (adds it to sys.modules and executes its body), but binds only the name f in the current module namespace:

  • import a in module b -> Imports the module in file a.py as a

  • from b import f in module a -> ImportError: cannot import name f (because the first execution of from b import f did not get to see the definition of the function object f in module b)

In this last case, the from ... import ... itself fails with an error because the interpreter knows earlier in time that you are trying to access something in that module which does not exist. Compare it to the first AttributeError, where the program did not see any problem until it tried to access attribute f (in the expression b.f).

The double execution problem of the code in the main module

When importing the module in the file used to start the program (imported as __main__ first) from another module, the code in that module gets executed twice and any side effects in that module execution will happen twice too. This is why it is not recommended to import the main module of the program again in other modules.

Using sys.modules to confirm my conclusions above

I will show how checking the contents of sys.modules can clarify this problem:

a.py

import sys

assert '__main__' in sys.modules.keys()
print(f'{__name__}:')
print('\ta imported:', 'a' in sys.modules.keys())
print('\tb imported:', 'b' in sys.modules.keys())

import b

b.f()

b.py

import sys

assert '__main__' in sys.modules.keys()
print(f'{__name__}:')
print('\ta imported:', 'a' in sys.modules.keys())
print('\tb imported:', 'b' in sys.modules.keys())

import a

assert False  # Control flow never gets here

def f():
    print('hi from b.f()')

The output of python3 a.py:

__main__:
    a imported: False
    b imported: False
b:
    a imported: False
    b imported: True
a:
    a imported: True
    b imported: True
Traceback (most recent call last):
  File "a.py", line 8, in <module>
    import b
  File "/home/andrei/PycharmProjects/untitled/b.py", line 8, in <module>
    import a
  File "/home/andrei/PycharmProjects/untitled/a.py", line 10, in <module>
    b.f()
AttributeError: module 'b' has no attribute 'f'