Search code examples
cmemorystatic

Why doesn't the size of the binary file increase with the size of the static data?


When I compile the following C code with gcc -O0,

#include <stdlib.h>
#include <stdio.h>

char bytes[1000*1000*1000];

int main(void) {
    for (int i = 0; i < 1000*1000; i++)
        bytes[i] = (char) i;
    printf("%c\n", bytes[56789]);
    return EXIT_SUCCESS;
}

The binary is only 16kB. Where are the 1GB I allocated? It's not on the heap because I don't call malloc, and not on the stack because it's too much.


Solution

  • Because the area is initialized with zeroes, it doesn't need to be in the binary on some systems. The loader will simply tell the OS that certain parts of the memory space is to be marked allocated, and the OS will provide the necessary zeroed pages.

    After compiling your program (as a) using gcc on Linux on x86_64, you can see a "directive" to Allocate 1,000,000,032 (3b9aca20) bytes. NOBITS refers to the fact that it's not in the binary.

    $ readelf -S a
    There are 31 section headers, starting at offset 0x3998:
    
    Section Headers:
      [Nr] Name              Type             Address           Offset
           Size              EntSize          Flags  Link  Info  Align
    ...
      [26] .bss              NOBITS           0000000000004020  00003010
           000000003b9aca20  0000000000000000  WA       0     0     32
    ...
    Key to Flags:
      W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
      L (link order), O (extra OS processing required), G (group), T (TLS),
      C (compressed), x (unknown), o (OS specific), E (exclude),
      l (large), p (processor specific)
    

    Not only does it not take space in the binary, it might take very little RAM as well.

    On a system with virtual memory (such as your desktop, tablets and phones), it might actually be possible to allocate a zero-initialized static array far larger than the system's actual RAM. It is possible on such a system because the OS only needs to start spending real resources on it once it starts being modified. And even then, it might only need to spend resources for the specific memory pages (e.g. 4 KiB sections) that were modified.

    You'd still be limited by the process's address space, but that's practically unlimited on a 64-bit system.