Running unittest with modules that must import other modules

Question

Our Python 3.10 unit tests are breaking when the modules being tested need to import other modules. When we use the packaging techniques recommended by other posts and articles, either the unit tests fail to import modules, or the direct calls to run the app fail to import modules. The other posts and articles we have read do not show how to validate that both the application itself and the unit tests can each import modules when called separately. So we created a bare bones example below and are asking how to structure the packaging correctly.

What specific changes must be made to the syntax below in order for the two Python commands given below to successfully run on the bare bones example app given below?

Problem description

A Python 3.10 app must import modules when called either directly as an app or indirectly through unit tests.

Packages must be used to organize the code.

Calls to unit tests are breaking because modules cannot be found.

The two test commands that must run without errors to validate solution of this problem are:

C:\path\to\dir>python repoName\app\first.py

C:\path\to\dir>python -m unittest repoName.unitTests.test_example

We have reviewed many articles and posts on this topic, but the other sources failed to address our use case, so we have created a more explicit example below to test the two types of commands that must succeed in order to meet the needs of this more explicit use case.

App structure

The very simple structure of the app that is failing to import packages during unit tests is:

repoName
  app
    __init__.py
    first.py
    second.py
    third.py
  unitTests
    __init__.py
    test_example.py
  __init__.py

Simple code to reproduce problem

The code for a stripped down example to reproduce the problem is as follows:

The contents of repoName\app\__init__.py are:

print('inside app __init__.py')
__all__ = ['first', 'second', 'third']

The contents of first.py are:

import second as second
from third import third
import sys

inputArgs=sys.argv

def runCommands():
  trd = third() 
  if second.something == 'platform':
    if second.another == 'on':
      trd.doThree()
  if second.something != 'unittest' :
    sys.exit(0)

second.processInputArgs(inputArgs)
runCommands()

The contents of second.py are:

something = ''
another = ''
inputVars = {}

def processInputArgs(inputArgs):
    global something
    global another
    global inputVars
    if ('unittest' in inputArgs[0]):
      something = 'unittest'
    elif ('unittest' not in inputArgs[0]):
      something = 'platform'
      another = 'on'
    jonesy = 'go'
    inputVars =  { 'jonesy': jonesy }

The contents of third.py are:

print('inside third.py')
import second as second

class third:

  def __init__(self):  
    pass

  #@public
  def doThree(self):
    print("jonesy is: ", second.inputVars.get('jonesy'))

The contents of repoName\unitTests\__init__.py are:

print('inside unit-tests __init__.py')
__all__ = ['test_example']

The contents of test_example.py are:

import unittest

class test_third(unittest.TestCase):

  def test_doThree(self):
    from repoName.app.third import third
    num3 = third() 
    num3.doThree()
    self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()

The contents of repoName\__init__.py are:

print('inside repoName __init__.py')
__all__ = ['app', 'unitTests']

Error resulting from running commands

The command line response to the two commands are given below. You can see that the call to the app succeeds, while the call to the unit test fails.

C:\path\to\dir>python repoName\app\first.py
inside third.py
jonesy is:  go

C:\path\to\dir>python -m unittest repoName.unitTests.test_example
inside repoName __init__.py
inside unit-tests __init__.py
inside app __init__.py
inside third.py
E
======================================================================
ERROR: test_doThree (repoName.unitTests.test_example.test_third)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\path\to\dir\repoName\unitTests\test_example.py", line 15, in test_doThree
    from repoName.app.third import third
  File "C:\path\to\dir\repoName\app\third.py", line 3, in <module>
    import second as second
ModuleNotFoundError: No module named 'second'

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (errors=1)

What specific changes must be made to the code above in order for all the modules to be imported correctly when either of the given commands are run?

Answer 1

Creating an "alias" for modules

Update the contents of repoName\app\__init__.py to:

print('inside app __init__.py')
__all__ = ['first', 'second', 'third']

import sys

import repoName.app.second as second
sys.modules['second'] = second

import repoName.app.third as third
sys.modules['third'] = third

import repoName.app.first as first
sys.modules['first'] = first

How to ensure first.py gets run even when imported

So when the test fixture imports repoName.app.third, Python will recursively import the parent packages so that:

import repoName.app.third is equivalent to

import repoName
# inside repoName __init__.py
import app
#inside app __init__.py
import third
#inside third.py

So running from repoName.app.third import third inside test_doThree, executes repoName\app\__init__.py. In __init__.py, import repoName.app.first as first is called. Importing first will execute the following lines at the bottom of first.py

second.processInputArgs(inputArgs)
runCommands()

In second.processInputArgs, jonesy = 'go' is executed setting the variable to be printed out when the rest of the test is ran.