Search code examples
gitlabgitlab-ciplaywrightplaywright-java

Using Multiple tags to run playwright tests in Gitlab

I have a large number of tests, all tagged with relevant tags. If I want to run a number of different tags in the same run, locally, I use

npx playwright test -g 'tag1 | tag2 | tag3'

and that works and runs all tests tagged with tag1, tag2, or tag3.

in Gitlab, I'm storing the "tags" in a CI/CD variable called Tags. so the command I have setup in my gitlab-ci.yml file is

npx playwright test -g $Tags

It picks up the tags properly, but it does not run the proper tests. Depending on how I format the variable

tag1|tag2|tag3

or

'tag1|tag2|tag3'

or

'tag1\|tag2\|tag3'

or any number of other variations, it either finds no tests, or just runs all the tests ignoring the tags entirely.

What's the magic configuration I need to use for this to work properly?

Klendathu

Solution

  • Okay, so after an afternoon trying too many iterations, I found that this finally worked:

    npx playwright test -g $Tag

    Where the variable $Tag was a NON-quoted string of tags separated by a pipe symbol.

    tag1|tag2|tag3

    If I used

    npx playwright test --grep $Tag

    That would fail. Not sure why it didn't like the full --grep command, but it just didn't work. Having the single quotes around the string of tags worked just fine locally, but it choked on it in Gitlab.