Convert a large ArrayBuffer into hexadecimal string representation in JavaScript

Question

I have an ArrayBuffer with content that needs to be converted to a hex-string, that should be sent in a JSON message. The data can be up to 2 GB. For this I would like to have a string that works as a dynamic array, so I efficiently could push_back individual characters. In C++, we have std::string (or std::vector), I think Java has a StringBuilder. What is the way of doing it in JavaScript?

I tried += but that starts to take minutes after reaching 128 MB. I guess this is because it results in an O(n^2) algorithm.

Note: I also need to do the inverse operation.

Answer 1

Well, why not use a temporary Uint8Array, like so:

function to_hex_string(bytes)
{
    let tmp = new Uint8Array(2*bytes.length);
    for(let k = 0; k != bytes.length; ++k)
    {
        let value = bytes[k];
        let msb = value >> 4;
        let lsb = value & 0xf;
        tmp[2*k] = msb < 10 ? msb + 48 : (msb - 10) + 65;
        tmp[2*k + 1] = lsb < 10 ? lsb + 48 : (lsb - 10) + 65;
    }

    return new TextDecoder().decode(tmp);
}

This outperforms joining an array. Though it fails when trying to convert the array to a string when trying 512 MiB. I guess there is some hardcoded limit. Maybe a JS process may not use more than 4 GiB because of a 32-bit VM.