Search code examples
cucumbercypressgherkincypress-cucumber-preprocessor

Cypress Gherkin should not stop suite execution on error in afterEach-hook

Context: I have a Cypress project that uses cypress-cucumber-preprocessor. In the feature file I define multiple scenarios. Each scenario performs an action on the page under test and takes a screenshot (sometimes multiple screenshots). I use an afterEach hook to then compare all screenshots taken in the scenario to their respective base image.

Feature: Example
Scenario 1
Given logged user
When goes to homepage
Then take a screenshot
# Now a hook compares the screenshot to the base

Scenario 2
Given logged user
When user makes a deposit
Then take a screenshot
# Now a hook compares the screenshot to the base

Scenario 3
... etc. ...

The hook is implemented in the file where all the then steps for the different visual validations are implemented.

Example:
Then('Then take a screenshot', () => {...})
Then('Then take a screenshot of element xyz and blackout the background', () => {...})

// The hook is implemented in this same file
afterEach(() => {
    cy.checkVisualResults();
});

Expected behaviour: I need Cypress to continue with the next scenario in the feature file, if there is a failure in the previous scenario.

Observed behaviour: Cypress stops the execution of the 'suite' (by which it means feature file) if a comparison of a screenshot to its base image fails after one of the scenarios. The error is this: Because this error occurred during a after each hook we are skipping the remaining tests in the current suite.

Maybe I need to put the hook in another place. From what I observe is that it is executed after each scenario, not after each screenshot is taken. I have other feature files that don't use scenarios and just take multiple screenshots. Before I added the hook Cypress would directly compare the current to the base image and stop running the remaining steps in those feature files. With the hook in place and the capture and comparison separated I see that for those types of feature files it works. What I need now is to adapt it to work for feature files that use scenarios.

Solution

  • Can you make another step instead of using the afterEach() hook?

    Feature: Example
Scenario 1
Given logged user
When goes to homepage
Then take a screenshot
Then compare the screenshot to the base

    Step

    Then('Then compare the screenshot to the base', () => {
  cy.checkVisualResults()
})

    or incorporate in the screenshot step

    Then('Then take a screenshot', () => {
  cy.screenshot()
  cy.checkVisualResults()
})

    Both approaches should isolate the failure to the specific failing test.

    Substitute Cucumber After()

    The After() hook behaves differently wrt failure - docs here

    Scenario hooks

    Before() and After() is similar to Cypress' beforeEach() and afterEach(), but they can be selected to conditionally run based on the tags of each scenario, as shown below.

    Furthermore, failure in these hooks does not result in remaining tests being skipped. This is contrary to Cypress' beforeEach and afterEach.