Search code examples
jenkinsjunitbehat

Behat, JUnit and Jenkins


I'm going a little insane trying to interpret Behat results as JUnit reports.

I'm using an ant build in order to generate tests results, this is how it is invoked:

<target name="behat" description="Run Behat Scenarios">
    <exec executable="php">
        <arg value="vendor/behat/behat/bin/behat" />
        <arg value="--format" />
        <arg value="pretty" />
        <arg value="--out" />
        <arg value="std" />
        <arg value="--format" />
        <arg value="junit" />
        <arg value="--out" />
        <arg value="${basedir}/build/logs/" />
    </exec>
</target>

This is the console output from Jenkins:

INFO: Processing JUnit
INFO: [JUnit] - 1 test report file(s) were found with the pattern 'build/logs/default.xml' relative to '/var/lib/jenkins/workspace/Web-Analysis' for the testing framework 'JUnit'.
WARNING: The file '/var/lib/jenkins/workspace/Web-Analysis/build/logs/default.xml' is an invalid file.
WARNING: At line 4 of file:/var/lib/jenkins/workspace/Web-Analysis/build/logs/default.xml:cvc-complex-type.3.2.2: Attribute 'status' is not allowed to appear in element 'testcase'.
WARNING: At line 5 of file:/var/lib/jenkins/workspace/Web-Analysis/build/logs/default.xml:cvc-complex-type.3.2.2: Attribute 'status' is not allowed to appear in element 'testcase'.
ERROR: The plugin hasn't been performed correctly: The result file '/var/lib/jenkins/workspace/Web-Analysis/build/logs/default.xml' for the metric 'JUnit' is not valid. The result file has been skipped.

And this is the generated JUnit file:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="default">
  <testsuite name="Testing" tests="2" skipped="0" failures="0" errors="0" time="0">
    <testcase name="Home Page" status="passed" time="0"></testcase>
    <testcase name="Dashboard is locked to guests" status="passed" time="0"></testcase>
  </testsuite>
</testsuites>

I've seen a few questions related to it, but couldn't really find an answer on how to solve this. Any help would be appreciated, and I'd be happy to give more details if needed! I'm using Behat 3.5.0 if that helps.

--edit Just adding behat.yml and console output

default:
    gherkin:
        cache: ~
    extensions:
        Laracasts\Behat\ServiceContainer\BehatExtension: ~
        Behat\MinkExtension\ServiceContainer\MinkExtension:
            default_session: laravel
            laravel: ~



behat:
     [exec] Feature: Testing
     [exec]     In order to learn Behat
     [exec]     As a developer
     [exec]     I want to learn how to install and create features
     [exec] 
     [exec]   Scenario: Home Page                  # features/example.feature:6
     [exec]     Given I am on the homepage         # FeatureContext::iAmOnHomepage()
     [exec]     Then I should see "Audio" # FeatureContext::assertPageContainsText()
     [exec] 
     [exec]   Scenario: Dashboard is locked to guests # features/example.feature:10
     [exec]     When I go to "admin"                  # FeatureContext::visit()
     [exec]     Then the url should match "login"     # FeatureContext::assertUrlRegExp()
     [exec]     And I can do something with Laravel   # FeatureContext::iCanDoSomethingWithLaravel()
     [exec] 
     [exec] 2 scenarios (2 passed)
     [exec] 5 steps (5 passed)
     [exec] 0m0.14s (22.99Mb)

Solution

  • I just had this problem after upgrading Jenkins xUnit plugin to 2.2.3 (from 2.0.0), downgrading to 2.0.0 "solved" the problem.