What's the maximum precision (after the decimal point) of a float in Javascript

Question

An algorithm I'm using needs to squeeze as many levels of precision as possible from a float number in Javascript. I don't mind whether the precision comes from a number that is very large or with a lot of numbers after the decimal point, I just literally need as many numerals in it as possible.

(If you care why, it is for a drag n' drop ranking algorithm which has to deal with a lot of halvings before rebalancing itself. I do also know there are better string-based algorithms but the numerical approach suits my purposes)

The MDN Docs say that:

The JavaScript Number type is a double-precision 64-bit binary format IEEE 754 value, like double in Java or C#. This means it can represent fractional values, but there are some limits to what it can store. A Number only keeps about 17 decimal places of precision; arithmetic is subject to rounding.

How should I best use the "17 decimal places of precision"?

Does the 17 decimal places mean "17 numerals in total, inclusive of those before and after the decimal place"

e.g. (adding underscores to represent thousand-separators for readability)

# 17 numerals: safe
111_222_333_444_555_66

# 17 numerals + decimal point: safe
111_222_333_444_555_6.6
1.11_222_333_444_555_66

# 18 numerals: unsafe
111_222_333_444_555_666

# 18 numerals + decimal point: unsafe
1.11_222_333_444_555_666
111_222_333_444_555_66.6

I assume that the precision of the number determines the number of numerals that you can use and that the position of the decimal point in those numerals is effectively academic.

• Am I thinking about the problem correctly?
• Does the presence of the decimal point have any bearing on the calculation or is it simply a matter of the number of numerals present
• Should I assume that 17 numerals is safe / 18 is unsafe?
• Does this vary by browser (not just today but over say, a 10 year window, should one assume that browser precision may increase)?

Answer 1

Short answer: you can probably squeeze out 15 "safe" digits, and it doesn't matter where you place your decimal point.

It's anyone's guess how the JavaScript standard is going to evolve and use other number representations.

Notice how the MDN doc says "about 17 decimals"? Right, it's because sometimes you can represent that many digits, and sometimes less. It's because the floating point representation doesn't map 1-to-1 to our decimal system.

Even numbers with seemingly less information will give rounding errors.

For example 0.1 + 0.2 => 0.30000000000000004

console.log(0.1 + 0.2);

However, in this case we have a lot of margin in the precision, so you can just ask for the precision you want to get rid of the rounding error

console.log((0.1 + 0.2).toPrecision(1));

For a larger illustration of this, consider the following snippet:

for(let i=0;i<22;i++) { 
  console.log(Number.MAX_SAFE_INTEGER / (10 ** i)); 
}

You will see a lot of rounding errors on digit 16. However, there would be cases where even the 16th decimal shows a rounding error. If you look here

https://en.wikipedia.org/wiki/IEEE_754

it states that binary 64 has 15.95 decimal digits. That's why I'd guess that 15 digits is the max precision you will get out of this.

You'd have to do your operations, and before you save back the number to any representational form, you'd have to do .toPrecision(15).

Finally this has some good explanations. https://floating-point-gui.de/formats/fp/

BTW, I got curious by reading this question so I read up as I wrote this answer. There are many people with better knowledge of this than me.