Search code examples
playwright

Playwright - How can parallel tests handle a bottleneck?

I am having some trouble running some Playwright tests in parallel. The challenge is that each test has three step, and only the 1st and 3rd steps can run in parallel. The 2nd step must be taken after all tests have finished stage 1 and before all tests can begin step 3.

Here's a little diagram of the bottleneck. Diagram of bottleneck

I've tried doing the following 3-step plan:

  1. Run a bunch of tests in parallel (retrying individual tests if necessary)
  2. Run one test all on its own (retrying it if necessary)
  3. Run a bunch more tests in parallel using data generated by step 1 (retrying if necessary)

I can do steps 1 and 2. What I can't figure out is how to pass information (like record ID#s and some expected values) from the step 1 tests to the step 3 tests. Every time a test fails and Playwright retries that test, all the stored information is erased.

Should I try making the step 1 tests send their output to a csv file that the step 3 tests could read?

Is there another better option for handling the bottleneck?

Solution

  • I figured out a solution. There are two parts.

    Part 1 of Solution: Passing data from test to test.

    I made the stage 1 tests write JSON files with the information needed in later tests. Just make sure each scenario has a unique filename that the later test will also have access too.

    Then I made the stage 3 tests read those JSON files to get the needed information. I set those tests to fail immediately if they could not find, read, or parse, the file contents.

    After I did that, the only issue was making sure all the Stage 1 tests finished before starting Stage 2 (and that Stage 2 finished before starting Stage 3).

    Part 2 of Solution: Controlling Test order

    I set up projects (https://playwright.dev/docs/api/class-testproject) in the config file. The below set up allowed me to run some tests using the default schedule and other tests in the specific order I required

      projects: [
{
  // Everything NOT in a project (ie: all normal tests)
  name: 'default',
  grepInvert: /@project/,
},

// Projects to do Setup Actions
{
  // Do setup tasks before running the actual tests
  name: 'setup',
  grep: /@projectSetup/,
  teardown: 'teardown',
},

// Projects to do Teardown Actions
{
  // Delete or clean up test records after running the tests
  name: 'teardown',
  grep: /@projectTeardown/,
},


// Projects to run stage #1
// Run these tests in parallel using: test.describe.configure({ mode:'parallel'});
{
  // Create the records that will be updated and verified in later stages
  name: 'DoStage1',
  grep: /@projectDoStage1/,
  dependencies: ['setup']
},


// Projects to run stage #2
// Put the test(s) in one describe block and run the test(s) one at a time using: test.describe.configure({ mode:'default'});
{
  // Do the Stage 2 task (that impacts all records created as part of stage 1)
  name: 'DoStage2',
  grep: /@projectDoStage2/,
  dependencies: ['DoStage1'],
},

// Projects to run stage #3
// Run these tests in parallel using: test.describe.configure({ mode:'parallel'});
{
  // Do the Stage 3 task (that verify all records created in stage 1 and updated in stage 2)
  name: 'DoStage3',
  grep: /@projectDoStage3/,
  dependencies: ['DoStage2'],
},

    ]