Search code examples
pythonflaskpopen

Python popen exception on cwd during flask request


There are two main steps:
The first is a flask endpoint:

@app.route('/updater/client', methods=['GET'])
def update_client():
    try:
        return _update(request, Unit.CLIENT)
    except:
        sv_exceptions.get_and_log_exception_info()
        return Response(status=BAD_REQUEST)

The _update() method is 'heavy'. It checks github repository, makes diff between the two revisions, forms a zip archive and sends it back as a response. It may take some time.
The issue that is that there is one of the steps that finds a latest revision in the repository via popen:

def get_latest_version(unit: Unit):
    repository_path = unit.repository_path
    branch = unit.branch

    print(f'repository_path: {repository_path}')
    try:
        with subprocess.Popen(["git", "rev-parse", branch],
                              cwd=r'{}'.format(repository_path),
                              universal_newlines=True,
                              stdout=subprocess.PIPE,
                              stderr=subprocess.PIPE) as p:
            for line in p.stdout:
                return line.rstrip('\r\n')
    
    except Exception as e:
        print(f'Exception in popen: {e}')
    return None

Everything is ok when the first request finishes before the second request:

repository_path: ./../../../client-repository
127.0.0.1 - - [02/Sep/2021 13:04:44] "GET /updater/client?os=windows&version=e7369fc9 HTTP/1.1" 200 -
repository_path: ./../../../client-repository
127.0.0.1 - - [02/Sep/2021 13:04:46] "GET /updater/client?os=linux&version=e7369fc9 HTTP/1.1" 200 -

And the exception when one of the requests starts before the second finishes

repository_path: ./../../../client-repository
127.0.0.1 - - [02/Sep/2021 13:04:49] "GET /updater/client?os=windows&version=e7369fc9 HTTP/1.1" 400 -
repository_path: ./../../../client-repository
Exception in popen: [Errno 2] No such file or directory: './../../../client-repository'
expected str, bytes or os.PathLike object, not NoneType
127.0.0.1 - - [02/Sep/2021 13:04:50] "GET /updater/client?os=linux&version=e7369fc9 HTTP/1.1" 200 -

As you can see the repository_path is always present but in the second case it falls with the No such file or directory.
I don't understand why does it happen.


Solution

  • Answering on my own question:

    It works when I replace the relative path ./../../../client-repository with the absolute path /home/user/client-repository.
    I may be wrong but I guess that because of the concurrency Popen tries to apply cwd one more time when it is already inside the folder from the previous execution.