Search code examples
c++opencvvisual-c++rounding-error

x64 rounding inconsistency in cvRound() (_mm_cvtsd_si32)


On x64 Windows using MSVC2013, I am using the cvRound function of OpenCV with the intention of round up from x.5 values. I've come across an inconsistency where cvRound(17.5f) returns 18 (good!), but cvRound(20.5f) returns 20 and not 21 as expected

cvRound is simply implemented thus, so it seems to be an Microsoft inconsistency in _mm_cvtsd_si32().

int  cvRound( double value )
{
    __m128d t = _mm_set_sd( value );
    return _mm_cvtsd_si32(t);
}

Can anyone suggest how/why this could be?

FWIW, cvRound(20.5f + 1e-3f) returns 21.


Solution

  • The rounding behavior of the SSE instructions is configurable via the floating point environment (specifically, the MXCSR register). There are several IEEE rounding modes. The default rounding mode is round-to-nearest, ties-to-even, so if the value is exactly in the middle of two representable values, the result is rounded to the nearest even value.

    Consider the following test program that demonstrates the different rounding modes in action:

    #include <fenv.h>
    #include <immintrin.h>
    #include <stdio.h>
    
    int main()
    {
        printf("Default:        %d\n", _mm_cvtsd_si32(_mm_set_sd(20.5)));
        fesetround(FE_DOWNWARD);
        printf("FE_DOWNWARD:    %d\n", _mm_cvtsd_si32(_mm_set_sd(20.5)));
        fesetround(FE_UPWARD);
        printf("FE_UPWARD:      %d\n", _mm_cvtsd_si32(_mm_set_sd(20.5)));
        fesetround(FE_TONEAREST);
        printf("FE_TONEAREST:   %d\n", _mm_cvtsd_si32(_mm_set_sd(20.5)));
        fesetround(FE_TOWARDZERO);
        printf("FE_TOWARDZERO:  %d\n", _mm_cvtsd_si32(_mm_set_sd(20.5)));
    }
    

    Output:

    Default:        20
    FE_DOWNWARD:    20
    FE_UPWARD:      21
    FE_TONEAREST:   20
    FE_TOWARDZERO:  20