override malloc and new calls in cpp program

Question

I want to call my custom malloc/new calls instead of standard library functions. 1. for malloc, I have this short program

    #include <stdio.h>
    #include <stdlib.h>

    void *__real_malloc(size_t size);

    void *__wrap_malloc(size_t size)
    {
        void *ptr = __real_malloc(size);
        printf("malloc(%ld) = %p\n", size, ptr);
        return ptr;
    }

    int main(void) {
      int* ptr = (int*) malloc(sizeof(int));
      return 0;
    }

I use the following command to compile g++ -Wl,--wrap,malloc main.cpp But getting below error

/usr/bin/ld: /tmp/ccnB04KY.o: in function `__wrap_malloc(unsigned long)':
b3.cpp:(.text+0x18): undefined reference to `__real_malloc(unsigned long)'
/usr/bin/ld: /tmp/ccnB04KY.o: in function `main':
b3.cpp:(.text+0x54): undefined reference to `__wrap_malloc'
collect2: error: ld returned 1 exit status

This works with gcc for .c files but not with g++ and .cpp files. What's the issue?

2. Also I cant figure out how to overide new calls?

Answer 1

Why does g++ not work with -Wl,--wrap,malloc?

g++ implies C++, and C++ ABI is different from C ABI. So you need to add extern "C" around __real_malloc and __wrap_malloc:

extern "C" {
void *__real_malloc(size_t size);

void *__wrap_malloc(size_t size) {
  void *ptr = __real_malloc(size);
  printf("malloc(%ld) = %p\n", size, ptr);
  return ptr;
}
}

To avoid conflicts, C++ compilers rewrite names to "random" (but it's not) strings. This is called name mangling and it encodes information about namespaces, function arguments (because you can overload functions in C++ and linkers require unique names for each one of them), etc. In comparison, C compilers translate the name to the assembly without much change. This Compiler Explorer example shows that x86 msvc compiler inserts a _ before the function name, while the other 3 compilers (x64 msvc, x86-64 gcc, and x86-64 clang) keeps the original name defined in C.

-Wl,--wrap,malloc is an option passed to the linker, which asks the linker to look for a function named __wrap_malloc. With extern "C", g++ generates names as if the function is defined in a C source file so the linker can find it.

How to override new?

You can override new specifically for a class, or provide a global new. See https://en.cppreference.com/w/cpp/memory/new/operator_new.