Search code examples
simgrid

Is it possible to spawn big amount of processes in SimGrid?

I have master process which spawns 50000 processes

int master(int argc, char* argv[])
{
  for (int i = 0; i < 10000; i++){
    string name = tostring(i);
    MSG_process_create(name.c_str(), some_code, NULL, MSG_host_self());
  }
  return 0;
}

But when I run this code I have an error:

[0.000000] /home/ubuntu/Downloads/simgrid/src/simix/smx_context.cpp:187: [xbt/CRITICAL] Failed to protect stack: Cannot allocate memory

How can I overcome it or it is impossible?

Solution

  • From http://simgrid.gforge.inria.fr/simgrid/3.13.90/doc/options.html#options_virt:

    If you want to push the scalability limits of your code, you might want to reduce the contexts/stack-size item. Its default value is 8192 (in KiB), while our Chord simulation works with stacks as small as 16 KiB, for example.

    Also, you want to make sure (with valgrind) that your application does not leak memory because that's what you need the most to scale in amount of processes.