Search code examples
c++stackconstantsstack-overflow

How many const variables can I declare before running out of memory?


I am writing code that has a lot of const variables (mostly ints and enums) and I am wondering, is there a maximum number of variables I can declare? My idea is that these const variables are allocated on the stack, which means I can declare about 1MB/4bytes = 250000 variables (which is plenty), assuming the stack has a size of 1MB. Am I correct?

A quick example of what i mean:

Test.cpp:

const unsigned int VECTOR_ID = 4;
const unsigned int MATRIX_ID = 3;

int main()
{
  std::cout << VECTOR_ID << " : " << MATRIX_ID << std::endl;
  return 0;
}

Solution

  • Note that constants which are known at compile time may not correspond to any objects; when you compile with optimization enabled the constants will be compiled directly into the machine instructions as immediate values. Here is a simple example. This means that there is no limit to the number of constant variables as such. It also means that constants which are not used at all will likely disappear completely. (And if they are used in any non-trivial way the size of the code will exceed the size of the data.)

    Even if indeed your constant variables become objects, e.g. because their address is taken, they will be "compiled into your program" and be a part of the executable.

    A program's size, and the size of its segments, is limited by the format of the executable, the system resources, and potentially by the build tools. An Intel page seems to indicate that static data (where global constants could end up) is limited to 2 GB under Windows even on 64 bit architectures (which is still three orders of magnitudes larger than your use case):

    Note that the limit on static and stack data is the same in both 32-bit and 64-bit variants. This is due to the format of the Windows Portable Executable (PE) file type, which is used to describe EXEs and DLLs as laid out by the linker. It has 32-bit fields for image section offsets and lengths and was not extended for 64-bit variants of Windows. As on 32-bit Windows, static data and stack share the same first 2GB of address space.

    A quick search seemed to indicate that this limit is not present in a modern Linux.

    Independently of any binary limits, the build tools (compiler and linker) may have more restrictive limits. Even if a later optimizer stage eliminates all constants, their definitions still need to be parsed. There may be limits to the number of names in a scope, or in a translation unit.

    • Microsoft C++ has a limit of 6144 for the number of member initializers, for example.
    • The C standard specifies a minimum of 511 identifiers per scope, but most compilers will allow for more.
    • The C++20 standard (public drafts here, the numbers are the same) specifies recommended minimums for a number of quantities in Annex B. Among them are
      • 65 536 external identifiers in one translation unit
      • 1024 identifiers with block scope declared in one block
      • 16 384 non-static data members (including inherited ones) in a single class
      • 1024 static data members of a class
      • 4096 enumeration constants in a single enumeration.

    I just created one data point about a C++ program with 2 million external variables and 2 million local variables:

    • gcc: Compiles successfully with an x86_64-pc-msys gcc 13.2. I had to explicitly set the stack size though, by passing -Wl,--stack,20000000 on the command line (so that it became gcc -O0 -Wl,--stack,20000000 -o many2 many2.cpp; the 20000000 was a safe guess). Without that setting, the executable was produced but crashed at run time. The source code is 101,555,599 bytes long, the executable only 64,028,280 bytes. Compilation takes a bit more than one minute on my pretty average PC.
    • MSCV: cl.exe has been running with 8% CPU (half a core) for a couple hours now. The command line was cl /O0 many2.cpp.

    Note to self: Don't specify -Wall because each unused variable generates a warning on the terminal. Printing millions of lines on a mintty under msys takes longer than I was willing to wait (extrapolating, it must be hours).