I have the following piece of code that I run with the latest (2.4.0) version of Catch2:
#include "catch.hpp"
#include <iostream>
TEST_CASE("Test") {
int x = 0;
SECTION("A") {
std::cout << "A";
++x;
REQUIRE(x == 1);
}
SECTION("B") {
std::cout << "B";
++x;
REQUIRE(x == 1);
}
std::cout << "X\n";
REQUIRE(x == 1);
}
If I run this, everything works as expected and I get:
AX
BX
=================================================================
All tests passed (4 assertions in 1 test case)
Clearly, the test case is run twice, once for each section.
If I change the assert in one of the sections, say to REQUIRE(x == 0)
, again, everything works as expected, Catch2 runs each section once and tells me the first one failed. However, if I change the assert to REQUIRE(x == 0)
in both sections, the result is confusing (I have shortened it slightly):
A
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.out is a Catch v2.4.0 host application.
Run with -? for options
---------------------------------------------------------------------
Test
A
---------------------------------------------------------------------
test.cpp:10: FAILED:
REQUIRE( x == 0 )
with expansion:
1 == 0
B--------------------------------------------------------------------
Test
B
---------------------------------------------------------------------
test.cpp:15: FAILED:
REQUIRE( x == 0 )
with expansion:
1 == 0
X
---------------------------------------------------------------------
Test
---------------------------------------------------------------------
test.cpp:19: FAILED:
REQUIRE( x == 1 )
with expansion:
0 == 1
=====================================================================
test cases: 1 | 1 failed
assertions: 3 | 3 failed
The test is clearly run three times, the last time bypassing both of the sections. Is this behaviour expected? I have tried looking at the Catch2 docs but could not find anything relevant.
From the docs:
One leaf section is executed on each run through a TEST_CASE. The other sections are skipped. Next time through the next section is executed, and so on until no new sections are encountered.
So the problematic test has to be run 3 times.