Alright, so I'm trying to learn reverse engineering and x64 assembly. As an example I wrote this little test program in C:
#include <stdio.h>
int square(int num) {
return num * num;
}
int main() {
int ans = square(5);
printf("%d", ans);
}
This results in the following assembly code for the square
function:
push rbp
mov rbp,rsp
mov DWORD PTR [rbp-0x4],edi
mov eax,DWORD PTR [rbp-0x4]
imul eax,eax
pop rbp
ret
This seems kind of weird to me, as edi
is the function argument, which I believe is then stored on the stack and loaded back into eax
, where it is then multiplied? Why not skip lines 3 and 4 and just imul edi, edi
?
So I opened up radare2 and did just that, but now the program returns seemingly random numbers, I'm guessing memory addresses?
Can someone explain to me why GCC uses this seemingly redundant register, and what I did wrong trying to patch the binary?
You didn't enable optimizations, so you're looking at debug code. With -O3
, the output is:
One note about your question, you need to assign to eax
, because eax
stores the return value in an x86 application.