Search code examples
c#roundingrounding-error

How do deal with infinitely repeating numbers as decimals?


When division results in an infinitely repeating number, the number obviously gets truncated to fit into the size of the decimal. So something like 1/3 becomes something like 0.3333333333333333333. If we then multiply that number by 3, we get something like 0.999999999999999999 rather than 1 as we would get if the true value of the fraction had been preserved.

This is a code sample of this from the MSDN article on decimal:

decimal dividend = Decimal.One;
decimal divisor = 3;
// The following displays 0.9999999999999999999999999999 to the console
Console.WriteLine(dividend/divisor * divisor); 

This causes an issue when the value 0.9999999999999999999 is compared with 1 for equality. Without the loss of precision they would be equal, but of course in this case the comparison would result in false.

How do people typically deal with this problem? Is there a more elegant solution other than defining some margin of error to every comparison?


Solution

  • This is a very old and widely known numeric computation problem. You already said you are looking for a solution other than defining some margin of error to every comparison. An approach that comes to my mind is to build the mathematic expression tree in memory first and do the calculation last. Having this at hand we can do some simplifications using some known rules before doing the calculations. Rules such as:

    • Remove equal numbers in numerator and denominator of a fraction if they are not zero
    • Square root of a square of a number is the number itself
    • ...

    Therefore instead of storing 1/3 in a decimal/double value which equals 0.33333... we can store an instance of Fraction(1, 3). Then we can define all other expressions like this to build an expression rather than doing the calculations. In the end we can first simplify the expression with the rules above and then calculate the result.

    I searched the web briefly to find libraries to do that and didn't find any yet. But I'm sure some can be found for other languages/platforms or even .NET.

    Please note that the above approach in the end just makes a better result and yet does not solve this problem inherent in the nature of numeric computations.