Search code examples
c++catch-unit-testcatch2

How to compare floating point in catch2

I am using Catch v2.13.1

What is the correct way to compare float values. I thought the below would fail, but both pass.

REQUIRE(1147332687.7189338 == Approx(1147332688.4281545).margin(0.0001));
REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

However this fails as expected

REQUIRE(abs(1147332687.7189338 - 1147332688.4281545) <= Approx(0).margin(0.0001));

I don't understand why the first two statements wouldn't work

Solution

  • There are a couple of things to consider in the posted example.

    REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

    This will pass, "unexpectedly". The reason can be found in the documentation (assertions - floating point comparisons).

    Approx is constructed with defaults that should cover most simple cases. For the more complex cases, Approx provides 3 customization points:

    • epsilon - epsilon serves to set the coefficient by which a result can differ from Approx's value before it is rejected. By default set to std::numeric_limits<float>::epsilon()*100.
    • [...]

    In the posted example, the two numbers differ by a cofficient near 6.2e-10, while the default is (given a 32-bit float) near 1.2e-5.

    The following test wouldn't pass.

    CHECK( a == Approx(b).epsilon(1e-12) );

    The other tests involve margin, which is described in the documentation as

    • margin - margin serves to set the the absolute value by which a result can differ from Approx's value before it is rejected. By default set to 0.0.

    The caveat, though, can be found in issue#1507.

    This is because of the default value for epsilon in the Approx class and the fact that Approx::equalityComparisonImpl will pass if the value is in the range of the margin OR the epsilon values.

    So, this test wouldn't pass:

    CHECK( a == Approx(b).margin(0.0001).epsilon(1e-12) );

    Note that this "issue" seems to be marked as resolved - not a bug:

    So, I don't think this is a bug.

    The reason for this is that it is very hard (well, impossible), to determine the user's intent, thus it is better to assume that the user has set up both checks correctly -- after all, if the user does not want a relative comparison, they can always set the epsilon to zero. In fact, I think that using both tolerances and taking the most forgiving one is the superior option to implement things like the Approx Matcher (#1499).