I have a GitLab CI job running a series of Postman requests using a custom environment. I'm using Newman to run them alongside the newman-reporter-htmlextra npm plugin to generate a test report.
The job looks like the following:
postman-tests:
stage: postman-tests
image:
name: wojciechzurek/newman-ci
before_script:
- cd ci/tests/postman
- npm install -g newman-reporter-htmlextra
script:
- newman run Non-regression_tests.postman_collection.json -e Tests.postman_environment.json \
--reporters htmlextra --reporter-htmlextra-export newman-results.html
- ls -la # Check report generation
artifacts:
when: always
paths:
- newman-results.html
allow_failure: true
When I run newman on my mac (newman 4.5.0), the requests and associated tests run properly and the report is generated. However, the job fails and the report is not generated:
$ newman run Non-regression_tests.postman_collection.json -e Tests.postman_environment.json --reporters htmlextra --reporter-htmlextra-export newman-results.html --color
Uploading artifacts...
WARNING: newman-results.html: no matching files
ERROR: No files to upload
ERROR: Job failed: exit code 1
It seems that the issue may be caused by the testing series in itself rather than the report generation, as the job fails even when I don't generate the report.
I tried different runners: Docker with official newman images, SSH and shell over machines where I had installed newman (version 4.5.6) and the htmlextra reporter beforehand. All fail.
It's interesting to note that the tests series and report generation both succeed when run locally on the machines behind the SSH and shell runners, but they do fail when launched from GitLab CI.
What did I forget/do wrong that prevents the test report generation from GitLab CI?
My .yml
for testing, look like this - It's very basic but I've just run it again and it was running fine:
stages:
- test
newman_tests:
stage: test
image:
name: postman/newman_alpine33
entrypoint: [""]
script:
- newman --version
- npm install -g newman-reporter-htmlextra
- newman run collection.json -e environment.json --reporters cli,htmlextra --reporter-htmlextra-export testReport.html
artifacts:
when: always
paths:
- testReport.html
One thing that I do have is entrypoint: [""]
in the image
block.