Search code examples
pythonvirtualenvsalt-project

Efficiently Synchronizing Python Project Environment Across Multiple Hosts with SaltStack


I'm currently working on a Python project named 'test' located in /home/user/projects on a salt-master host. I've successfully used file.recurse to distribute this project to three different hosts. The project runs within a virtual environment created by pyenv. Now, I'm facing a challenge in synchronizing the project's virtual environment across these hosts. Initially, I considered using file.recurse to sync the entire .pyenv directory to the other hosts, but this approach is resource-intensive. I also thought about packaging the entire .pyenv, but I'm not convinced it's an optimal solution.

Is there a more efficient method to distribute the project while simultaneously synchronizing the project environment? I'm looking for a solution that avoids excessive resource usage and ensures that the Python environment is consistent across all hosts. Any advice or alternative approach would be greatly appreciated!


Solution

  • Create a descriptor of the project dependencies (e.g. requirements.txt, pyproject.toml, Pipfile, etc.) and then use Salt to run the command to create it on each target.

    e.g. with virtualenv.managed:

    /opt/myproject/venv:
      virtualenv.managed:
        - requirements: salt://myproject/requirements.txt
    

    Depending on Salt and Python versions, you may need to workaround e.g. issue #51742.