In our current c-project, we use python scripts for support and testing purposes such as unit testing, integration testing, benchmarking and communication.
Current folder structure (most files not shown):
.
|-- workingcopy1
|-- |-- config
|-- |-- |-- __init__.py
|-- |-- |-- parameters_one.py
|-- |-- |-- parameters_two.py
|-- |-- |-- parameters_three.py
|-- |-- src
|-- |-- |-- main_application_c_code.c
|-- |-- tests
|-- |-- |-- tools
|-- |-- |-- |-- display_communication_activity.py
|-- |-- |-- run_all_unit_tests.py
|-- |-- tools
|-- |-- |-- script1.py
|-- |-- |-- script2.py
|-- |-- |-- script3.py
|-- |-- utils
|-- |-- |-- python_utils
|-- |-- |-- |-- __init__.py
|-- |-- |-- |-- communication_utils.py
|-- |-- |-- |-- conversion_utils.py
|-- |-- |-- |-- constants.py
|-- |-- |-- |-- time_utils.py
|-- workingcopy2
...
|-- workingcopy3
...
Some python files are intented to be executed as script files ($ python script1.py) and some are intended to be included as modules in other python files.
What we would like to achive is a structure that enables us to have parameter and utility functions that can be used by:
It should also be possible to have several workingcopies checked out
Up until this date, all scripts have following lines at top:
import os, sys
current_path = os.path.abspath(os.path.dirname(__file__))
sys.path.append(os.path.join(current_path, '..', 'utils')) # Variates depending on script location in file tree
sys.path.append(os.path.join(current_path, '..', 'config')) # Variates depending on script location in file tree
import python_utils.constants as constants
import config.parameters_one as parameters
With about 20+ script files this has become hard to maintain. Is there any better way to achive this?
You should convert your folders into Python modules by adding an empty __init__.py file into it.
Also, you can add the Python shebang, so they are executable without explicitly calling the Python command from shell (Should I put #! (shebang) in Python scripts, and what form should it take?).
Once your folders are modules, you have to add only the main source path and you will be able to import the children modules in an easier manner. Furthermore, you should use a virtual environment (virtualenv) that can handle the paths for you (http://docs.python-guide.org/en/latest/dev/virtualenvs/) (and maybe virtualenvwrapper that allows you extra functionality)
I wanted to add a couple of additional strategies you could use here: One of the cool things about python is that everything is an object so you could import and pass your script modules as a variable to a function that run's them, initialising the appropriate path. Also, the function could "discover" the scripts by looking into the folder and walking through it.
Again, all this can be easily handled from pre-post activate virtualenvwrapper hooks.