Search code examples
.netf#base-class-library

Apparent F#/BCL floating point bug


The following is in FSI:

> System.Math.Round(0.2916, 2);;
val it : float = 0.29
> it * 100.;;
val it : float = 29.0
> int it;;
val it : int = 28

The result is the same everywhere I tried - a compiled F# 3.1/.NET 4.0 application, FSI in Visual Studio 2013 and 2015, .NET Fiddle, @fsibot....

Surely this is a bug somewhere, isn't it? What's going on here?


Solution

  • This is just how floating point numbers work. The number that appears as 29 in the output is actually slightly smaller than 29 (because floating point numbers are not precise):

    > (System.Math.Round(0.2916, 2) * 100.0) - 29.0;;
    val it : float = -3.552713679e-15