Redis server and its fork() - when the vm_enough_memory is called?

Question

I'm trying to understand how the fork() is working in the Redis server running on Linux and how Redis will generate the fork: Cannot allocate memory response.

From what I inverstigated I see the next:

1 redis-server calls fork() in its rdbSaveBackground():

if ((childpid = fork()) == 0) {

2 This calls a fork() from the glibc's sysdeps/nptl/fork.c (which seems to be in the /usr/lib/libc.so.6):

$ ldd /usr/bin/redis-server
    linux-vdso.so.1 (0x00007ffde8d93000)
    libjemalloc.so.2 => /usr/lib/libjemalloc.so.2 (0x00007fa5da60b000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fa5da4c5000)
    libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fa5da4c0000)
    libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fa5da49f000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fa5da2dc000)

But I'm not sure the ldd is the right way to check it. The ltrace only displays the fork() itself - but not the source from where it was called.

3 sysdeps/nptl/fork.c executes the arch_fork macro

4 And then sysdeps/unix/sysv/linux/arch-fork.h calls exactly the clone() Linux syscall:

ret = INLINE_SYSCALL_CALL (clone, flags, 0, NULL, 0, ctid);

Which can be seen in a strace's output:

accept(5, {sa_family=AF_INET, sin_port=htons(60816), sin_addr=inet_addr("127.0.0.1")}, [128->16]) = 6
...
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2097, ...}) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7ff26beda190) = 1790

If so - when then vm_enough_memory is executed?

It must rise the Redis' famous "Can't save in background: fork: Cannot allocate memory".

I was able to find the vm_enough_memory in the Linux's fork() syscall:

The security_vm_enough_memory_mm:

        if (security_vm_enough_memory_mm(oldmm, len)) /* sic */
            goto fail_nomem;

Desribed in the include/linux/security.h:

static inline int security_vm_enough_memory_mm(struct mm_struct *mm, long pages)
{
    return __vm_enough_memory(mm, pages, cap_vm_enough_memory(mm, pages));
}

So, the questions are (yeah, I know it's bad post two Q's in the same topic, but they are tied):

1. What is the correct way to see library functions calls by a process (besides the ltrace and ldd I used)

2. What about glibc's fork() and Linux clone()? How would Redis produce the fork: Cannot allocate memory if it is really uses glibc's fork()?

Maybe I'm looking in the wrong direction, and Redis will call the Linux fork() syscall? If so - this will explain everything (but not the strace's output with the clone()...).

Answer 1

Redis server and its fork() - when the vm_enough_memory is called?

Redis doesn't directly call vm_enough_memory - it simply calls the fork wrapper in glibc which in turn calls into kernel's fork system call (which is clone system call on all modern Linux kernels).

What is the correct way to see library functions calls by a process (besides the ltrace and ldd I used)

ltrace is fine to get the library calls. ldd lists the shared library dependencies but doesn't tell you which library a given function resides. You may find nm and objdump utilities useful in that regard.

If so - when then vm_enough_memory is executed?

vm_enough_memory is a kernel function - strace or other tools will not list them. strace is usually good enough (unless you are the debugging the kernel itself) for applications to find out which system calls are being called by the user code and/or C library.

For example, strlen function is defined (it's in the text section, so you know it's defined there) in the C library:

 $ objdump -T  /lib/x86_64-linux-gnu/libc.so.6 | grep strlen
000000000007fd10 g   iD  .text  000000000000003d  GLIBC_2.2.5 strlen

(Read man page for various options).

What about glibc's fork() and Linux clone()? How would Redis produce the fork: Cannot allocate memory if it is really uses glibc's fork()?

Again, you are complicating things. Redis simply calls fork() and if it returns -1 (fork failure) then it simply prints that error message based on the errno value.

That particular message comes from this line of code:

   serverLog(LL_WARNING,"Can't save in background: fork: %s",
            strerror(errno));

When fork fails, childpid is -1 and the errno is set by it indicating the error. When errno is set to ENOMEM, you get Cannot allocate memory error message.

Here's a simple example demonstraing it:

#include <stdio.h>
#include <string.h>
#include <errno.h>

int main(int argc, char** argv) {
    errno = ENOMEM;
    printf("errno (ENOMEM): %s\n", strerror(errno));
}

And it outputs:

$ gcc -Wall -std=c11 test.c
$ ./a.out
errno (ENOMEM): Cannot allocate memory