Caching playwright browser binaries in bitbucket pipelines

Question

My goal is to enable sharding for Playwright on Bitbucket Pipelines, so I want to use parallel steps along with caching. My bitbucket-pipelines.yml script looks like this:

image: mcr.microsoft.com/playwright:v1.25.0-focal

definitions:
  caches:
    npm: $HOME/.npm
    browsers: ~/.cache/ms-playwright #tried $HOME/.cache/ms-playwright ; $HOME/ms-playwright ; ~/ms-playwright
  steps:
    - step: &base
        caches:
          - npm
          - node
          - browsers
    - step: &setup
        script:
          - npm ci
          - npx playwright install --with-deps
    - step: &landing1
        <<: *base
        script:
          - npm run landing1
    - step: &landing2
        <<: *base
        script:
          - npm run landing2
    - step: &landing3
        <<: *base
        script:
          - npm run landing3

pipelines:
  custom:
    landing:
      - step: *setup
      - parallel:
          - step: *landing1
          - step: *landing2
          - step: *landing3

Besides trying various location for the caches definition I also tried to just set repo variable PLAYWRIGHT_BROWSERS_PATH to 0 and hope that browsers will appear within node modules.

Solution with caching browsers within default location leads to this (in all 4 cases mentioned in comment of the file):

While not caching browsers separately and using PLAYWRIGHT_BROWSERS_PATH=0 with caching node also does not work, each parallel step throws an error saying browser binaries weren't installed.

I also tried varying between npm install and npm ci, exhausting all of the solutions listed here.

I hope somebody has been able to resolve this issue specifically for Bitbucket Pipelines, as that is the tool we are currently using in the company.

Answer 1

You can NOT perform the setup instructions in a different step than the ones that need very that setup. Each step runs in a different agent and should be able to complete regardless of the presence of any cache. If the caches were present, the step should only run faster but with the same result!

If you try to couple steps through the cache, you loose control on what is installed: node_modules will quite often be an arbitrary past folder not honoring the package-lock.json for the git ref where the step is run.

Also, your "setup" step does not use the caches from the "base" step definition, so you are not correctly polluting the listed "base" caches either. Do not fix that, it would cause havoc to your life.

If in need to reuse some setup instructions for similar steps, use yet another YAML anchor.

image: mcr.microsoft.com/playwright:v1.25.0-focal

definitions:

  caches:
    npm: ~/.npm
    browsers: ~/.cache/ms-playwright

  yaml-anchors: # nobody cares what you call this, but note anchors are not necessarily steps

    - &setup-script >-
        npm ci
        && npx playwright install --with-deps

    # also, the "step:" prefixes are dropped by the YAML anchor
    # and obviated by bitbucket-pipelines-the-application
    - &base-step
        caches:
          - npm
          # - node  # don't!
          - browsers

    - &landing1-step
        <<: *base-step
        script:
          - *setup-script
          - npm run landing1

    - &landing2-step
        <<: *base
        script:
          - *setup-script
          - npm run landing2

    - &landing3-step
        <<: *base
        script:
          - *setup-script
          - npm run landing3

pipelines:

  custom:
    landing:
      - parallel:
          - step: *landing1-step
          - step: *landing2-step
          - step: *landing3-step

See https://stackoverflow.com/a/72144721/11715259

---

Bonus: do not use the default node cache, you are wasting time and resources by uploading, storing and downloading node_modules folders that will be wiped by npm ci instructions.