Search code examples
pythonpython-3.xpython-poetry

Developing Python external lib in realtime with Poetry not working

I have a project that uses a library stored in a separate repository. This library my-lib contains some features that are used by my application. I want to link the lib with my main project to develop it in realtime. In other words, I want to start my main project and edit the lib in the background causing the project to rebuild on changes. I have spent some time trying to figure it out, but no luck.

This what I did:

  1. Suppose I have a Repo folder that contains both, the project and the lib:
/Repo/my-lib/
/Repo/my-app/backend/
  1. Inside my-app I did poetry add ../../my-lib (according to THIS), which added an entry in dependencies inside pypoetry.toml file:
my-lib = { path = "../../my-lib" }
  1. I modified this entry manually to:
my-lib = { path = "../../my-lib", develop = true }
  1. Started my project and edited the lib.

Unfortunately, changes in my-lib are not reflected in the project. Even trying poetry update my-lib does not help.

So I checked .venv/site-packages folder. I wanted to know if my-lib is a symlink or physical copy of the folder and it is the 2nd one. I would expect that some kind of linking should happen here. I searched through many issues and topics in the Web and for me it looks like it should work...

The project I'm working on (especially lib part) is very hard and needs much of debugging all the time, so removing .venv/site-packages/my-lib and reinstalling my-lib over and over again is really annoying and time consuming.

Maybe, my folder structure is a problem here? I have virtualenv inside my project folder, so the whole project structure looks like:

/Repo/my-lib/
  pyproject.toml

/Repo/my-app/backend/
  .venv/
    lib
      python3.8/
        site-packages/
          my-lib/
  pyproject.toml

I'm using the newest version of Poetry@1.1.13 and it was installed using their script (not via pip).

Solution

  • The solution I ended up finally is using:

    my-lib = { path = "../../my-lib", develop = true }

    But I also do some additional steps that work always:

    1. Delete .venv folder
    2. poetry cache clear --all .
    3. poetry lock --no-update
    4. poetry install

    After these steps everything always works fine.