Search code examples
javascriptnode.jsswigv8node-gyp

Passing JavaScript array to c/c++ library and back efficiently


Hi basically what I want to do is passing a JavaScript array to a c module function, then the function modify the array in place, then JavaScript reads the modified array.

Current approach is use carrays.i and array_functions, create and converting Array to and from doubleArray and due to copying array, its giving me result worse than native JS. My array have about 41000 items.

C module: ~10ms(actual C function running time ~0.1ms)

JS module: ~3ms

For me, it's not possible to use doubleArray from very beginning (as this is a part of a larger process). So the question is how can I improve it? Is it possible to use TypedArray/ArrayBuffer? If yes then how?

following is my pseudo code

let cArray = MyCModule.new_doubleArray(array.length),
    outArray = new Array(array.length);
arrayCopyJS2C(cArray, array);//written in JS and use a lot of time
MyCModule.MyCFunction(cArray, array.length);
arrayCopyC2JS(cArray, outArray);//also written in JS and use a lot of time

Solution

  • Yes, using an ArrayBuffer (with externalized backing store) is an efficient way to share a (number) array between JavaScript and C, because it doesn't require you to copy things around. That's assuming that you can use a TypedArray "from the beginning" on the JavaScript side; if the same limitation applies as to using a doubleArray from the beginning and you'd still have to copy, then the benefit will be smaller or nonexistent (depending on how fast you've made accesses to your doubleArray).

    That said, V8 generates highly efficient code for operations on number arrays. I'm finding it hard to believe that the same function takes either 3ms in JS or 0.1ms in C. Can you share your JS implementation? If a C implementation is 30x as fast, then I bet the JS implementation could be improved a lot to get pretty close to that. Array operations are usually dominated by the time it takes to actually get the elements from memory, and no language has a particular advantage at that.