Search code examples
pythongitvirtualenvcontinuous-deploymentpython-poetry

How to manage editable shared libraries with poetry?


I have 2 python projects A and B

  • A depends on B
  • B is a shared library that is used by other projects
  • B takes a long time to build and install

Previously, when I have to set up my development environment on a fresh computer the workflow was:

  1. create a virtual environment
  2. git clone A and B
  3. run B's setup develop
  4. run A's setup develop

Now I can edit A and B and push the changes to git without having to rerun setup on A and B every time.

Now, I want to integrate poetry into my workflow. Ideally I want to be able to git clone A and run poetry install A and have it mirror my above setup and but I am having trouble installing B through A's setup while still keeping B editable. Based on this thread it seems like sharing venv between two project is not possible.

My stopgap workaround is:

  • git clone A and B
  • poetry install B
  • edit A's pyproject.toml to point to the B directory that I manually installed (B = {path = "../B/"})
  • poetry install A

I really don't like this workflow since A's pyproject.toml now only works on my computer. I looked through issue threads and poetry documentation and could not find solution dealing with this situation. Any and all advises are greatly appreciated.

A's pyproject.toml

[tool.poetry.dependencies]
python = "^3.7"
# B = {git = "https://repo.com/myrepo/B.git"}
B = {path = "../B/"}

Edit: I realized that if I install B before A and edited pyproject before installing A I can prevent 2 installation of B


Solution

  • I'm not sure if this solves your problem but you might be able to use one venv for two projects (is that what you want?) by creating the virtual environment first and then install both project's dependencies in it.

    I can't verify it now but something like this might help:

    # create the virtual environment and activate  
    $ python3 -mvenv .venv
    $ . .venv/bin/activate
    
    # install dependencies of both projects in it
    $ cd B
    $ poetry install
    $ cd ../A
    $ poetry install