what is the aim of JS_CANONICALIZE_NAN in spidermonkey engine?

I wondering what is the aim of JS_CANONICALIZE_NAN and if it is always needed on all platforms ?

Solution

This is a fun one! So, SpiderMonkey internally uses a tagged value representation to represent JavScript's "untyped values" -- this allows the VM to determine things like "the variable stored in a is an number, and the value stored in b is an number, so running a + b does numerical addition".

There are a bunch of different schemes for value tagging, and SpiderMonkey uses one that's referred to as "NaN boxing". This means that all untyped values in the engine are represented by 64 bit values that can either be:

a double, or
a tagged non-double that lives in the "NaN space" of IEEE double-precision floating point values.

The real trick here is that modern systems use generally use a single bit pattern to represent NaN, which you can observe as the result of math.h's sqrt(-1) or log(0). but there are a lot of bit patterns which are also considered NaNs according to the IEEE floating point spec.

A double is composed of the sub-fields:

{sign: 1, exponent: 11, significand: 52}

NaNs are represented by filling the exponent field with 1s and placing a non-zero value in the significand.

If you run a little program like this to see your platform's NaN values:

#include <stdio.h>
#include <math.h>
#include <limits>

static unsigned long long 
DoubleAsULL(double d) {
    return *((unsigned long long *) &d);
}

int main() {
    double sqrtNaN = sqrt(-1);
    printf("%5f 0x%llx\n", sqrtNaN, DoubleAsULL(sqrtNaN));
    double logNaN = log(-1);
    printf("%5f 0x%llx\n", logNaN, DoubleAsULL(logNaN));
    double compilerNaN = NAN;
    printf("%5f 0x%llx\n", compilerNaN, DoubleAsULL(compilerNaN));
    double compilerSNAN = std::numeric_limits<double>::signaling_NaN();
    printf("%5f 0x%llx\n", compilerSNAN, DoubleAsULL(compilerSNAN));
    return 0;
}

You'll see output like this:

 -nan 0xfff8000000000000 // Canonical qNaNs...
  nan 0x7ff8000000000000
  nan 0x7ff8000000000000
  nan 0x7ff4000000000000 // sNaN (signaling)

Note that the only difference for the quiet NaNs is in the sign bit, always followed by 12 bits of 1s, satisfying the NaN requirement mentioned above. The last one, a signaling NaN, clears out the 12th (is_quiet) NaN bit and enables the 13th to keep the NaN invariant mentioned above.

Other than that, the NaN space is free to play in -- 11 bits to fill in the exponent, make sure the signficand is non-zero, and you've got a lot of space left. On x64 we use a 47 bit virtual address assumption, which leaves us 64 - 47 - 11 = 6 bits for annotating value types. On x86 all object pointers fit in the lower 32 bits.

However, we still need to make sure that non-canonical NaNs, if they creep in through something like js-ctypes, don't produce something like looks like tagged non-double values, because that could lead to exploitable behavior in the VM. (Treating numbers as objects is very-much-so bad news bears.) So, when we form doubles (like in DOUBLE_TO_JSVAL) we make sure to canonicalize all doubles where d != d to the canonical NaN form.

More info is in bug 584168.