We have a tox-enabled project (let's call it "main" project) which has a dependency on another tox project (let's call it "library" project) - all united in one repository since it is all part of a large overarching project.
How the project works for the regular user
For a regular install as an end-user you would simply install "library" first and then "main", right from the repository or whatever sources, and then run it.
What our issue is with tox
However, as a developer the situation is different, because "tox" should work and you might want to have multiple versions around at the same time.
You usually check out the large overarching repository and then the filesystem layout is this:
overarchingproject/main/
overarchingproject/main/src/
overarchingproject/main/tox.ini
overarchingproject/main/setup.py
...
overarchingproject/library/
overarchingproject/library/src/
overarchingproject/library/tox.ini
overarchingproject/library/setup.py
...
Now if I go into main/ and type "tox", this happens:
Current behavior: It will try to build the "main" project with the dependency on "library" - which will obviously result in an attempt to get "library" from pip. However, the project is not released yet (therefore not on pip) so it won't work - even though the lib is right there in the same repo.
Result: it doesn't work.
Workaround 1: We could set up our own package index/ask users to do that. However, asking everyone contributing to the project to do that with DevPI or similar just to be able to run the unit tests seems not such a good idea, so we'd need to do it centrally.
But if we provided a package index at some central place or a pip package for "library", people couldn't run the tests of "main" easily with involvement of a modified version of "library" they created themselves:
"library" is in the same repository after all, so people might as well modify it at some point.
That typing "tox" inside the "main" project folder will not easily pick up upon the present neighbouring "library" version but only a prepackaged online thing isn't exactly intuitive.
Workaround 2: We tried sitepackages=True and installing "library" in the system - however, sitepackages=True has caused us notable amount of troubles and it doesn't seem a good idea in general.
Desired behavior: We want tox to use the local version of the "library" in that folder right in the same overarching repository which people will usually get in one thing:
That version might be newer or even locally modified, so this is clearly what the dev user wants to use. And it exists, which cannot be said about the pip package right now.
Why do we want the overarching repository anyway with subprojects ("main", "library", ...) and not just one single project?
We develop a multi-daemon large project with many daemons for various purposes, with shared code in some libs to form a university course management system (which deals with forums, course management with possibility to hand in things, attached code versioning systems for student projects etc.).
It is possible to just use a subset of daemons, so it makes sense that they are separate projects, but still they are connected enough that most people would want to have most of them - therefore all of them in one repository.
The library itself is also suitable to be used for entirely different projects, but it is usually used for ours to start with - so that is where it is stuffed into the repository. So that means it is always around in the given relative path, but it has its separate tox.ini and unit tests.
TL;DR / Summary
So how can we make tox look for a specific dependency in another toxable project folder instead of just pip when installing a project?
Of course "main"'s regular setup.py install process shouldn't mess around with tox or searching the local disk: it should just check for one specific relative path, and then give up if that one is not present (and fall back to pip or whatever).
So best would be if the relative path could be somehow stored in tox.ini.
Or is this all just a pretty bad idea? Should we solve this differently to make our "main" project easily toxable with the latest local dev version of "library" as present in the local repository?
As suggested in the comments to the response of diabloneo it is possible to supply an install_command in the tox.ini file:
I used this to make a bash script that takes all the usual pip arguments, but then runs pip before with just pip install --editable="file://`pwd`/../path/to/neighbour/repo", and only then actually runs the regular pip install $@ afterwards with the arguments to the script (as would be passed by tox to pip directly). I then used this script with install_command instead of the regular default pip command.
With this two-step procedure it works fine :-)