malloc and free in D/Tango not freeing memory?

Question

here is a simple d/tango code in windows:

module d_test.d;

import tango.util.log.Trace;
import tango.core.Thread;
import tango.stdc.stdlib : malloc, free;

void main() {

    Trace.formatln("Checking in...");
    Thread.sleep(10);

    int total_n = (100 * 1000 * 1000) / int.sizeof; // fill mem with 100MB of ints
    int* armageddon = cast(int*)malloc(total_n * int.sizeof);

    for(int i = 0; i < total_n; ++i) {
        armageddon[i] = 5;
    }

    Trace.formatln("Checking in...");
    Thread.sleep(10);

    free(armageddon);
    armageddon = null;

    Trace.formatln("Checking in...");
    Thread.sleep(10);


}

when I run the program, memory stays low ~2MB, and when I allocate an array of 100MB to the pointer memory usage jumps to ~100MB, which is fine. However, after free memory is still (I'm looking at task manager) at 100MB to the end of the program.

I thought it might be to windows pagefile cache or something, so I've tried a simple C++ program:

#include <iostream>
#include <windows.h>

using namespace std;

int main() {

  Cout << "Checking in..." <<< endl;
  Sleep(10000);


  int total_n = (100 * 1000 * 1000) / sizeof(int);
  int* armageddon = (int*)malloc(total_n * sizeof(int));

  for(int i = 0; i < total_n; ++i) {
    armageddon[i] = 5;
  }

  Cout << "Checking in..." <<< endl;
  Sleep(10000);

  free(armageddon);
  armageddon = NULL;

  Cout << "Checking in..." <<< endl;
  Sleep(10000);


return 0;
}

I've compiled it with g++ and everything seems to work like it should. When program starts - memory usage ~900kb, after allocation ~100MB, after free ~1,2MB...

So, what am I doing wrong or is this a bug?

Answer 1

It is going to depend on how free is implemented. For some interesting reading, take a look at Doug Lea's allocator, which tries to be efficient at managing memory requests that can span a wide range of sizes. His primary concern is that the malloc and free calls be fast, and correct.

But he does mention the issue of returning memory to the OS, and discusses what hinders it (memory fragmentation) and what aids it (use of mmap, or in a less flexible manner, sbrk). Once you've read this article, you'll have a clearer idea of the trade-offs which lead to infrequent return of memory to the OS.