Search code examples
pipyamlgithub-actions

github action pip dependencies not working after found cache


I had wrote a workflow that included caching, following the docs of action/cache@v2. I'm using docker-compose to host my postgres db and doing tests with django framework(not in docker container).

Here are my steps:

runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python 3.8
      uses: actions/setup-python@v2
      with:
        python-version: '3.8'

    - name: Cache Dependencies
      uses: actions/cache@v2
      id: cache
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-
    - name: Install Dependencies
      if: steps.cache.outputs.cache-hit != 'true'
      run: |
        echo "Installing dependencies and caching them."
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Start containers
      run: |
        pip cache dir
        cp .env.example .env
        docker-compose -f "docker-compose.yml" up -d --build
    - name: Run Tests
      run: |
        sleep 20
        coverage run manage.py test -v 2 && coverage report
        flake8
    - name: Stop containers
      if: always()
      run: docker-compose -f "docker-compose.yml" down

my requirements.txt already included the correct package that why first time running the build without caching it's success without any problems. The problem is any build after the first one, after the cache is found:

Run actions/cache@v2   with:
    path: ~/.cache/pip
    key: Linux-pip-0066acd4cae425b2654d95aae80b2bb4fda1c99d40c709cc1fa8a1595759a2c5
    restore-keys: Linux-pip-
     env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.10/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.10/x64/lib Received 0 of 63168059 (0.0%), 0.0 MBs/sec Received 63168059 of 63168059 (100.0%), 33.1 MBs/sec Cache Size: ~60 MB (63168059 B) /usr/bin/tar --use-compress-program zstd -d -xf /home/runner/work/_temp/3f7aab5a-e81f-4039-889e-c0879488e8e8/cache.tzst
-P -C /home/runner/work/base.django/base.django Cache restored successfully Cache restored from key: Linux-pip-0066acd4cae425b2654d95aae80b2bb4fda1c99d40c709cc1fa8a1595759a2c5

The test returning error due to package from requirements.txt not found:

Run sleep 20
  sleep 20
  coverage run manage.py test -v 2 && coverage report
  flake8
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.10/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.10/x64/lib
/home/runner/work/_temp/62e260b4-7d93-4a7c-a3e1-e930c4ea36b1.sh: line 2: coverage: command not found
/home/runner/work/_temp/62e260b4-7d93-4a7c-a3e1-e930c4ea36b1.sh: line 3: flake8: command not found
Error: Process completed with exit code 127.

Even though i specified the same cache path as the cache document and it's correct with the pip document https://pip.pypa.io/en/stable/cli/pip_install/#caching ?

here what pip cache dir printing out:

/home/runner/.cache/pip

i don't know why the depedencies not found even though the cache path is correct


Solution

  • After a while, i found this article to cache the env python location instead and now the cache is working correctly

    - uses: actions/cache@v2
      with:
        path: ${{ env.pythonLocation }}
        key: ${{ env.pythonLocation }}-pip-${{ hashFiles('requirements.txt') }}
    

    I don't know why caching the entire python path worked but the cache dir didn't. Maybe someone can give a clear explanation in the future, for now this is what fixed for me