Search code examples
jenkinsbddpython-behave

In Jenkins job, behave tests stops after any failure

I have created a jenkins "freestyle" job, in which I am trying to run multiple BDD testing process. Following is the "commands" I have put in "Jenins/Build/execute shell" section: 

    cd ~/FEXT_BETA_BDD
    rm -rf allure_reports  allure-reports  allure-results 
    pip install behave
    pip install selenium
    pip install -r features/requirements.txt

    # execute features in plan section
    behave -f allure_behave.formatter:AllureFormatter -f pretty -o ./allure-reports 
        ./features/plan/*.feature

    # execute features in blueprint section
    behave -f allure_behave.formatter:AllureFormatter -f pretty -o ./allure-reports 
        ./features/blueprint/*.feature

What I have found is in Jenkins, if there is any test case intermittent failure, such message is shown in the Console Output: 

"
    ...
    0 features passed, 1 failed, 0 skipped
    0 scenarios passed, 1 failed, 0 skipped
    3 steps passed, 1 failed, 1 skipped, 0 undefined
    Took 2m48.770s
    Build step 'Execute shell' marked build as failure
    "

And the leftover test cases are skipped. But if I was to run the behave command on my local host directly, I don't get this type of behaviour. The failure will be detected and the remaining test cases continues till all are finished.

So How may I work around this issue in Jenkins ?

Thanks,

Jack

Solution

  • You may try the following syntax:

    set +e

# execute features in plan section
behave -f allure_behave.formatter:AllureFormatter -f pretty -o ./allure-reports 
        ./features/plan/*.feature || echo 'ALERT: Build failed while running the plan section'

# execute features in blueprint section
behave -f allure_behave.formatter:AllureFormatter -f pretty -o ./allure-reports 
        ./features/blueprint/*.feature || echo 'ALERT: Build failed while running the blueprint section'

# Restoring original configuration
set -e

    Note:

    • Goal of set -e is to cause the shell to abort any time an error occurs. If you will see your log output, you will notice sh -xe at the start of execution which confirms that Execute Shell in Jenkins uses -e option. So, to disable it, you can use +e instead. However, it's good to restore it once your purpose is fulfilled so that subsequent commands produce expected result.

    Ref: https://superuser.com/questions/1113014/what-would-set-e-and-set-x-commands-do-in-the-context-of-a-shell-script