After upgrading Electron from 1 to 10, the --max-old-space-size command can't set a value larger than 4GB

Question

My Problem
I upgraded my project from Electron 1 to 10 this month. I find that the following command is not working now. No code changes.

app.commandLine.appendSwitch('js-flags', '--max-old-space-size=6144')

Hardware Configuration: 64 bits, 32GB RAM, Windows 10

My Try

1. clone repo at Electron Demo.
2. add the following code.

const showMemory = () => {
    const mem = process.memoryUsage()
    const format = bytes => {
      return (bytes / 1024 / 1024).toFixed(4) + ' MB'
    }
    console.log('process: heapTotal ' + format(mem.heapTotal) + ', heapUsed ' + format(mem.heapUsed))
}
const arr = []
while(true) {
    arr.push(new Array(1000000))
    showMemory()
}

Electron >=v9.0.0: The max value I can set is 4GB. If I set 8GB, it is still 4GB. It can reduce but not increase.
Electron <=v8.5.5: Set a value larger than 4GB successfully. When to crash depends on the value I set.

My Research

1. V8 v8.0 brings Pointer Compression. It sounds like that V8 limits the heap size to 4 GB, so we can't increase it.
2. But I try to increase it in pure Node.js v14.17.6 (V8 v8.4.371.23), no Electron. It works. From this issues, Node.js disables pointer compression still now.

node --max-old-space-size=6144 index.js

3. So V8 in Electron limits the size, and at the same time V8 in Node.js doesn't? I'm not sure if I'm right. I wait for a definite answer.
4. If I need to increase the size, is downgrading Electron the only way now?

Answer 1

(V8 developer here.) Your analysis seems correct: Electron 9.0 enabled V8 pointer compression, and pointer compression implies that the maximum heap size is capped at 4GB (because compressed pointers use 32 bits per pointer).

Pointer compression is a build-time flag for V8. Embedders can freely choose whether to enable it or not, but the decision has to be made at build time, it can't be made at runtime. If you build your own Electron, you can change the configuration accordingly and recompile to turn off pointer compression. Node.js decided to disable pointer compression, which is why bigger heaps work there.

I suppose you could also ask the Electron project to provide prebuilt binaries without pointer compression, but I have no idea how likely they are to accommodate such a request.

Note that due to the memory-saving effects of pointer compression (which is why the feature exists!), a 4GB compressed heap is roughly equivalent to a 6GB uncompressed heap (the exact numbers depend on what your app is doing).