How does Node/V8 determine its max heap size?

I'm looking for a canonical answer here.

If I have an application that looks like this:

const  {getHeapStatistics} = require("v8");
console.log(getHeapStatistics())

I get:

{
  total_heap_size: 5275648,
  total_heap_size_executable: 262144,
  total_physical_size: 4997120,
  total_available_size: 4342409984,
  used_heap_size: 3760144,
  heap_size_limit: 4345298944, // 👈 4.046875GB
  malloced_memory: 163968,
  peak_malloced_memory: 58368,
  does_zap_garbage: 0,
  number_of_native_contexts: 1,
  number_of_detached_contexts: 0,
  total_global_handles_size: 8192,
  used_global_handles_size: 2240,
  external_memory: 1328787
}

Now I can control that heap_size_limit value by using the max-old-space-size node option..

If write an application that continually adds items to an array, it'll error when size of allocated data exceeds that value - node won't dynamically increase the max size.

It seems like 4GB is the default value, but is there a canonical answer for this? I can't find it anywhere.

Additionally - how does V8 behave when the max size is greater than the available system memory? What I'm thinking about is a context where multiple node processes are spawned, and perhaps when they started plenty of system memory was free, but as they've all continued that system memory is no longer free. Is this an interaction between node and the OS and the OS will simply not allow the new memory to be allocated, and V8 will accept this and continue with the smaller heap size?

To give a specific example - say I'm running Jest tests on a system with 16 cores and 32GB memory. Jest runs the tests in parallel - one test per core. If allow Node to use 4GB per process, what appears to happen is we have some tests run very slowly - my hypothesis is that earlier processes use their full 4GB memory and later processes have to make do with less, and thus take longer. Using max-old-space-size=1800 appears to solve this problem, but the concern is that now I might be 'leaving memory on the table' - where if there are some processes that don't actually need the full 2GB, it would be nice if the other processes could use it.

There are other questions on Stack Overflow that ask at this:

V8 Heap Size Limit (No answer)

Solution

(V8 developer here.)

How does Node/V8 determine its max heap size?

If you don't explicitly configure a maximum, V8 will choose one for you based on heuristics. These heuristics can and do change sometimes (they certainly have in the past, and likely will eventually change again), so whatever happens today isn't guaranteed to always remain that way.

The summary of the state of things right now is that on 64-bit systems (except Android), you get half the physical memory, up to a cap of 4 GiB. If you care about the details, you can start reading code at HeapSizeFromPhysicalMemory.

If you want a particular heap size maximum, the most reliable option is to configure it explicitly yourself, rather than relying on defaults. If you're not sure and don't want to worry about it, then trusting the defaults is fine (even without knowing what they are).

node won't dynamically increase the max size.

Well, it wouldn't be much of a limit if it was dynamically increased, would it? The actual heap size will be increased dynamically, the heap size limit is the maximum to which it is allowed to be increased.

how does V8 behave when the max size is greater than the available system memory?

It will attempt to allocate memory (because you told it that it could), and eventually crash when that fails (for lack of alternative). Depending on which code paths requests the allocation, you may or may not see an "emergency GC" before that out-of-memory crash.

Out-of-memory situations are notoriously hard to reason about. There are even some cases where the OS will at first permit a large memory allocation request from an application, but won't actually commit all of the pages, and will then kill the application once it writes enough data into the region it previously allocated (which requires the kernel to on-demand commit the pages). When that happens, it's entirely beyond the application's control; and the OS only does it because it has no alternative option.

There are other questions on Stack Overflow that ask at this:

One answer is so "canonical" that Node has even copied parts of it into its documentation. And it even answers the last part of your question about what happens when the limit is too high!