Search code examples
githubgithub-actionspuppeteerallurejest-puppeteer

Running tests in parallel on GitHub does not generate Allure report for all test file. Only generate for the last test file


When I run these tests locally or in sequence on GitHub it generate Allure Report but when I run it parallel using three different GitHub instance it generate only result for the last executed test file.

on:
  workflow_dispatch:
jobs:
  build:
    runs-on: ubuntu-latest
    env:
        STAGE_USER_NAME: ${{ secrets.STAGE_USER__EMAIL}}
        STAGE_USER_PASSWORD: ${{ secrets.STAGE_USER_PASSWORD}}

    strategy:
      matrix:
        node-version: [18.14.0]
        test-file:
                  - bprod.test.js
                  - login.test.js
                  - Facebook.test.js
 
    steps:
    - uses: actions/checkout@v3
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v3
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
    - run: npm install
      run: npx jest ${{ matrix.test-file }} jest --reporters default jest-allure --detectOpenHandles --forceExit --testTimeout=300000
      continue-on-error: true
   
    - run: npm run alluregenerate
    - name: Upload a Build Artifact
      if: always()
      uses: actions/upload-artifact@v2-preview
      with: 
          name: report-${{ github.run_number }}
          path: allure-report

Solution

  • When you name the artifact, chose a name that is less ambiguous 1 to not that easily risk to use the same name for different artifacts.

    For example, this can be achieved by using a date from the matrix context:

        - name: Upload a Build Artifact
          if: always()
          uses: actions/upload-artifact@v2-preview
          with: 
              name: report-${{ github.run_number }}-${{ matrix.test-file }}
              path: allure-report
    

    (artifact name appended with test-file from matrix context)

    Alternatively, give each test report a different name and make those reports the artifacts (e.g. junit xml files), then have a collecting step obtaining all reports and run allure on all of them (e.g. a directory, c.f.1).


    1. Each artifact behaves as a file share. Uploading to the same artifact multiple times in the same workflow can overwrite and append already uploaded files

      from: Uploading to the same artifact - upload-artifact - actions