I am trying to mock a function using -wrap=symbol for my unit testing and I find different behavior based on the GCC optimization flags passed
Consider the below example program
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
void * __wrap_malloc(size_t sz)
{
printf("Called %s", __func__);
return (void *) 0xDEADBEEF;
}
int main()
{
void *ptr;
ptr = malloc(10);
assert(0xDEADBEEF == (unsigned int)(uintptr_t)(ptr));
return 0;
}
When using optimization
❯ gcc -Os -Wl,-wrap=malloc 1.c -o opt
❯ ./opt
opt: 1.c:16: main: Assertion `0xDEADBEEF == (unsigned int)(uintptr_t)(ptr)' failed.
[1] 12546 abort ./opt
Without optimization
❯ gcc -Wl,-wrap=malloc 1.c -o without_opt
❯ ./without_opt
Called __wrap_malloc
I tried disassembling the executable
With optimization
0000000000400450 <main>:
400450: 48 83 ec 08 sub $0x8,%rsp
400454: b9 20 06 40 00 mov $0x400620,%ecx
400459: ba 10 00 00 00 mov $0x10,%edx
40045e: be ea 05 40 00 mov $0x4005ea,%esi
400463: bf ee 05 40 00 mov $0x4005ee,%edi
400468: e8 d3 ff ff ff callq 400440 <__assert_fail@plt>
40046d: 0f 1f 00 nopl (%rax)
Without optimization
000000000040053e <main>:
40053e: 55 push %rbp
40053f: 48 89 e5 mov %rsp,%rbp
400542: 48 83 ec 10 sub $0x10,%rsp
400546: bf 0a 00 00 00 mov $0xa,%edi
40054b: e8 c7 ff ff ff callq 400517 <__wrap_malloc>
400550: 48 89 45 f8 mov %rax,-0x8(%rbp)
400554: 48 8b 45 f8 mov -0x8(%rbp),%rax
400558: 3d ef be ad de cmp $0xdeadbeef,%eax
40055d: 74 19 je 400578 <main+0x3a>
40055f: b9 4e 06 40 00 mov $0x40064e,%ecx
400564: ba 10 00 00 00 mov $0x10,%edx
400569: be 0a 06 40 00 mov $0x40060a,%esi
40056e: bf 10 06 40 00 mov $0x400610,%edi
400573: e8 c8 fe ff ff callq 400440 <__assert_fail@plt>
400578: b8 00 00 00 00 mov $0x0,%eax
40057d: c9 leaveq
40057e: c3 retq
40057f: 90 nop
There is no call to wrap_malloc. It looks like it got optimized out. Then I tried adding a print statement
int main()
{
void *ptr;
ptr = malloc(10);
>> printf("0x%x\n", (uintptr_t)ptr);
assert(0xDEADBEEF == (unsigned int)(uintptr_t)(ptr));
return 0;
}
❯ gcc -Os -Wl,-wrap=malloc 1.c -o print_opt
❯ ./print_opt
Called __wrap_malloc0xdeadbeef
print_opt: 1.c:17: main: Assertion `0xDEADBEEF == (unsigned int)(uintptr_t)(ptr)' failed.
[1] 31575 abort ./print_opt
Now I see the wrapped function getting called. But the assertion failed
0000000000400450 <main>:
400450: 48 83 ec 08 sub $0x8,%rsp
400454: bf 0a 00 00 00 mov $0xa,%edi
400459: e8 f9 00 00 00 callq 400557 <__wrap_malloc>
40045e: bf 0a 06 40 00 mov $0x40060a,%edi
400463: 48 89 c6 mov %rax,%rsi
400466: 31 c0 xor %eax,%eax
400468: e8 c3 ff ff ff callq 400430 <printf@plt>
40046d: b9 48 06 40 00 mov $0x400648,%ecx
400472: ba 11 00 00 00 mov $0x11,%edx
400477: be 10 06 40 00 mov $0x400610,%esi
40047c: bf 14 06 40 00 mov $0x400614,%edi
400481: e8 ba ff ff ff callq 400440 <__assert_fail@plt>
400486: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1)
40048d: 00 00 00
Could some one help me understand why/how I am seeing this behavior?. Thanks in advance
With gcc, malloc() is treated as what's called a "built in" function that the optimizer is aware of and handles specially (It can make certain assumptions about its return value, for example, which lead it to automatically failing the assertion without needing a runtime test of it or even calling malloc() in the first place). If you add -fno-builtin-malloc to your compilation options, that special treatment is disabled:
$ gcc -Os -fno-builtin-malloc -Wl,-wrap=malloc 1.c -o opt
$ ./opt
Called __wrap_malloc
Side note: I added a trailing newline to your printf() output, because a program that produces textual output without one can cause odd effects in the calling shell - without it, the line was overwritten by my shell's prompt, for example. Or you might get a trailing % printed, or other behavior.