Search code examples
karateassertion

Is it possible to do soft-assertions in Karate

Is it possible to continue the execution of a test step even if one of the assert/match fails (also known as: soft-assertion) ?

Example

Scenario: Testing
* def detail = {"a":{"data":[{"message":["push","dash"]},{"message":["data","Test"]}]}}
* match detail contains {"a":{"data":[{"message":["push","dash"]}]}}
* print detail

Here match will fail but execution stop at that point.

Question

Is there a way to do a soft-assertion so that the next step gets executed?

Solution

  • EDIT in 2021 - a PR introducing a continueOnStepFailure flag was contributed by Joel Ramos here and is available in Karate 1.0 onwards. You can find more details here: https://stackoverflow.com/a/66733353/143475

    If you use a Scenario Outline each "row" is executed even if one fails.

    Scenario Outline: Testing
* def detail = { a: 1, b: 2, c: 3 }
* match detail contains <expected>

  Examples:
    | expected |
    | { a: 1 } |
    | { b: 2 } |
    | { c: 3 } |

    Note that the concept of "soft assertions" is controversial and some consider it a bad practice:

    a) https://softwareengineering.stackexchange.com/q/7823

    b) https://martinfowler.com/articles/nonDeterminism.html

    For those looking for a way to show all mis-matches between 2 JSON objects, see this: https://stackoverflow.com/a/61349887/143475

    And finally, since some people want to do "conditional" match logic in JS, see this answer also: https://stackoverflow.com/a/50350442/143475