submodule couldn't import parentmodule (python)

Question

These are the directories of my module:

mymodule
|-__init__.py
|-file1.py
|-file2.py
|-test
  |-__init__.py
  |-test_file1.py
  |-test_file2.py

test_file1.py contains this command: from .. import file1.

---

To run the tests I did this (on the command line): python3 -m unittest test.test_file1. (When I ran that command for the whole test directory it just told me "everything is fine" but didn't find my tests.)

The answer (also in the command line, of course) was (without a huge part of the stacktrace):

   File "/media/me/my_usb/backup me/myfolder/django projects/django-mymodule/mymodule/test/test_file1.py", line 1, in <module>
    from .. import file1
ValueError: attempted relative import beyond top-level package

What to do to fix this? What's best-practice with tests in multiple files?

---

Edit: I tried a few things as suggested, and that's what I have done:

1. I changed the tests direction (as suggested in the Hitchhikers guide to python. That's what it looks now:

   modulewrapper
   |-mymodule (with the 2 files in it)
   |-... (docs, readME and this stuff)
   |-tests
     |-test_file1.py
     |-test_file2.py
   

2. I imported mymodule after inserting the direction like this (I added this code in the beginning of every test file):

   import sys
   sys.path.insert(0, '../mymodule')
   import file1
   

---

I started the test as usual: python3 -m unittest test_file1 from the tests directory. What now happened is this: (first the relevant part of the stacktrace, then my guess):

File "/media/me/my usb/backup me/my folder/django projects/django-mymodule/tests/test_file1.py", line 4, in <module>
    import file1
  File "../mymodule/file1.py", line 4, in <module>
    from .file2 import MyClass1, MyClass2
SystemError: Parent module '' not loaded, cannot perform relative import

How to deal with that new problem? (or is it the same as before? It seems hardly to be a kind of best_practice to change working code to be able to run tests.)

---

Edit2: For now, I jusr replaced from .file2 import someclass by from file2 import someclass. Can this have any negative side effects?

Answer 1

For the module test.test_file1, test is your top-level package. You can't go outside that for relative imports; you can only reach other modules within the test package.

Either run python3 -m unittest mymodule.test.test_file1 (with your working directory set to the parent directory of mymodule) or add mymodule to your Python path and use from mymodule import file1.

Generally, best practice is to put a tests directory next to your project:

mymodule
|-__init__.py
|-file1.py
|-file2.py
tests
|-test_file1.py
|-test_file2.py

Note that there is no __init__.py file, but some test runners (specifically pytest, may require one).

This is how the Django project itself organises tests, as do most other Python projects (see, for example, click, requests or flake8). Use a test runner to discover tests, like nose.

You'd need to add your mymodule parent directory to the Python path for this to work, of course, so you may want to add

import os
import sys
here = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.dirname(here))

to the top of your test files, or to put that into a utils.py file you put in your tests directory.

Also see the Structure section of the Hitchhikers Guide to Python.