So I am experimenting with how to call C++ programs from ASM. It works on debug mode, but not so much on release mode.
I have assembly code that looks like this :
sub rsp, 30h
mov rcx, 1
mov rdx, 2
mov r8, 3
mov r9, 4
mov qword ptr [rsp + 20h], 5
mov qword ptr [rsp + 28h], 6
call MyProc
add rsp, 30h
ret
And the MyProc function looks like this :
extern "C"
{
*other irrelevant methods here*
void MyProc(int a, int b, int c, int d, int e, int f)
{
cout << "First & Last Param : " << a << " and " << f << endl;
}
}
In debug mode, it runs properly. On release mode, it fails with : Error 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF
Whats going on, and how do I fix this?
After some experimenting, all I need to do is to sub the stack pointer by any number greater or equal than the (number of parameters of the function * 8) bytes, but the number has to end with an 8 to keep the stack aligned (credit to Hans), so numbers like 28h for (< 5 param), 38h(6 - 7 param), 48h (8 - 9 param) etc. would work.
Not sure if I'm wrong about this, but the first 4 bytes of the stack are reserved, which is why I need to sub the stack by more than 32 bytes...