Search code examples
c#f#floating-pointlanguage-lawyernon-deterministic

Does F# suffer from same C# caveats on non-deterministic floating point calculation?


The result of C# floating-point code can lead to different results.

This question is not about why 0.1 + 0.2 != 0.3 and the inherent imprecision of floating point machine numbers.

It is rather linked to the fact that the same C# code, with the same target architecture (x64 for instance), may lead to different results depending on the actual machine / processor that is used.

This question is directly related to this one: Is floating-point math consistent in C#? Can it be?, in which the C# problem is discussed.

For reference, this paragraph in the C# specs is explicit about that risk :

Floating-point operations may be performed with higher precision than the result type of the operation. For example, some hardware architectures support an "extended" or "long double" floating-point type with greater range and precision than the double type, and implicitly perform all floating-point operations using this higher precision type. Only at excessive cost in performance can such hardware architectures be made to perform floating-point operations with less precision, and rather than require an implementation to forfeit both performance and precision, C# allows a higher precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects

Indeed we actually experienced an ~1e-14 order of magnitude difference in an algorithm using only double, and we are afraid that this discrepancy will propagate for other iterative algorithm that use this result, and so forth, making our results not consistently reproducible for different quality / legal requirement we have in our field (medical imaging research).

C# and F# share the same IL and common runtime, however, as far as I understand, it may be more something driven by the compiler, which is different for F# and C#.

I feel not savvy enough to understand if the root of the issue is common to both, or if there is hope for F#, should we take the leap into F# to help us solve this.

TL;DR

This inconsistency problem is explicitly described in the C# language specs. We have not found the equivalent in F# specs (but we may not have searched at the right place).

Is there more consistency in F# in this regard?

i.e. If we switch to F#, are we guaranteed to get more consistent results in floating-point calculations across architectures?


Solution

  • In short; C# and F# shares the same run-time and therefore does floating point number computations in the same way so you will see the same behavior in F# as in C# when it comes to floating point number computations.

    The issue of 0.1 + 0.2 != 0.3 spans most languages as it comes from the IEEE standard of binary floating pointing numbers, where double is an example. In a binary floating point number 0.1, 0.2 and so on can't be exactly represented. This is one the reason some languages support hex float literals like 0x1.2p3 which can be exactly represented as a binary floating point number (0x1.2p3 is equal to 9 btw in a decimal number system).

    Lots of software that relies on double internally like Microsoft Excel and Google Sheet employ various cheats to make the numbers look nice but often isn't numerically sound (I am no expert I just read a bit of Kahan).

    In .NET and many other languages there is often a decimal data type that is a decimal floating point numbers ensuring 0.1 + 0.2 = 0.3 is true. However, it doesn't guarantee that 1/3 + 1/3 = 2/3 as 1/3 can't be represented exactly in a decimal number system. As there is no hardware for support for decimal they tend to be slower, in addition the .NET decimal is not IEEE compliant which may or may not be a problem.

    If you have fractions and you have lots of clock cycles available you can implement a "big rational" using BigInteger in F#. However, the fractions quickly grows very large and it can't represents 12th roots as mentioned in the comment as outcomes of roots arecommonly irrational (ie can't be represented as rational numbers).

    I suppose you could preserve the whole computation symbolically and try to preserve exact values for as long as possible and then very carefully compute a final number. Probably quite hard to do correct and most likely very slow.

    I've read a bit of Kahan (he co-designed 8087 and the IEEE standard for floating point numbers) and according to one of the papers I read a pragmatic approach to detect rounding errors due to floating point number is to compute thrice.

    One time with normal rounding rules, then with always round down and finally with always round up. If the numbers are reasonably close at the end the computation is likely sound.

    According to Kahan cute ideas like "coffins" (for each floating point operation produce a range instead of single value giving the min/max value) just don't work as they are overly pessimistic and you end up with ranges that are infintely large. That certainly match my experience from the C++ boost library that does this and it's also very slow.

    So when I worked with ERP software in the past I have from what I read of Kahan recommended that we should use decimals to eliminate "stupid" errors from like 0.1 + 0.2 != 0.3 but realize that there are still other sources for errors but eliminating them is beyond us in compute, storage and competence level.

    Hope this helps

    PS. This is a complex topic, I once had a regression error when I changed the framework at some point. I dug into it and I found the error came from that in the old framework the jitter used the old-style x86 FPU instructions and in the new jitter it relied on the SSE/AVX instructions. There are many benefits by switching to SSE/AVX but one thing that was lost that the old style FPU instructions internally used 80 bits floats and only when the floating point numbers left the FPU they were rounded to 64 bits while SSE/AVX uses 64 bits internally so that meant the results differed between frameworks.