NoraUi - Scenario usage

Question

Is it possible to use the same scenario to test different entries, some of them will lead to "right" success and others to "right" failure? In case of right behavior, the test will be OK.
For example, a login connection should succeed with one data entry and should fail with another entry, but the test must end to success.

Answer 1

Yes this sample exist in CI of NoraUi:

https://github.com/NoraUi/NoraUi/blob/master/src/test/resources/data/in/hello.csv

author;zip;city;element;element2;date;title;Result
Jenkins T1;35000;Rennes;smile;smile;16/01/2050;;
Jenkins T2;75000;Paris;smile;smile;;;31
Jenkins T3;56100;Lorient;smile;smile;;;25
Jenkins T4;35000;Rennes;smile;smile;;;
Jenkins T5;35000;Rennes;noExistElement;noExistElement;;;42
Jenkins T6;35000;;smile;smile;;;2
Jenkins T7;35000;Rennes;;;;;4
Jenkins T8;;Rennes;;smile;smile;;58

You find the number of KO step in Result column (input data provider).

For example:

Jenkins T5;35000;Rennes;noExistElement;noExistElement;;;42

step N°42 of https://github.com/NoraUi/NoraUi/blob/master/src/test/resources/steps/hello.feature is KO with noExistElement

When I click on $bakery.DemoPage-<element>

AFTER in your CI:

https://noraui.github.io/#continuousIntegration

travis-ci online sample (unix batch sample use SED): https://github.com/NoraUi/NoraUi/blob/master/test/run.sh

echo "***************************************************"
echo "** Integration tests verification                **"
echo "***************************************************"

counters1=$(sed -n 's:.*\[Excel\] > <EXPECTED_RESULTS_1>\(.*\)</EXPECTED_RESULTS_1>.*:\1:p' nonaui.log | head -n 1)
echo "******** $counters1"
nb_counters1=$(sed -n ":;s/$counters1//p;t" nonaui.log | sed -n '$=')
echo "********" found $nb_counters1 times

counters2=$(sed -n 's:.*\[Excel\] > <EXPECTED_RESULTS_2>\(.*\)</EXPECTED_RESULTS_2>.*:\1:p' nonaui.log | head -n 1)
echo "******** $counters2"
nb_counters2=$(sed -n ":;s/$counters2//p;t" nonaui.log | sed -n '$=')
echo "******** found $nb_counters2 times"

# 3 = 1 (real) + 2 counters (Excel and CSV)
if [ "$nb_counters1" == "3" ] && [ "$nb_counters2" == "3" ]; then
    echo "******** All counter is SUCCESS"
else
    echo "******** All counter is FAIL"
    echo "$counters1 found $nb_counters1 times"
    echo "$counters2 found $nb_counters2 times"
    pwd
    ls -l
    cat target/reports/html/index.html
    exit 255
fi

echo "***************************************************"
echo "** Unit tests verification                       **"
echo "***************************************************"

counterFailures=$(sed -n 's/.*\[\(.*\)\] Tests run: \(.*\), Failures: \([1-9]\), Errors: \(.*\), Skipped: \(.*\), Time elapsed.*UT.*/\3/p' nonaui.log | head -n 1)
echo "******** counter Failures: $counterFailures"

counterErrors=$(sed -n 's/.*\[\(.*\)\] Tests run: \(.*\), Failures: \(.*\), Errors: \([1-9]\), Skipped: \(.*\), Time elapsed.*UT.*/\4/p' nonaui.log | head -n 1)
echo "******** counter Errors: $counterErrors"

if [ "$counterFailures" == "" ] && [ "$counterErrors" == "" ]; then
    echo "******** All unit test are SUCCESS"
else
    if [ "$counterFailures" != "" ]; then
        echo "******** At least one unit test is Failure"
    fi
    if [ "$counterErrors" != "" ]; then
        echo "******** At least one unit test is Error"
    fi
    exit 255
fi