I have a function in an unmanaged Win32 DLL that takes a variable number of arguments and therefore needs to be __cdecl rather than __stdcall or any other calling convention. At the moment I'm linking explicitly to the DLL (which is to say, I'm using LoadLibrary and GetProcAddress rather than linking to the DLL's .lib file).
I can call the function just fine, but since it's cdecl I need to add the following inline assembly instruction after each call:
retVal = addVecs(v1, v2, v3, v4);
__asm add esp, 64 ;Sizeof VECTOR struct is 16 bytes
printf("The sum of all these vectors is:\n\tMagnitude: %f\n\tDirection (radians): %f\n\n", retVal.mag, retVal.dir);
If I do not include the inline assembly then the program crashes after the call to addVecs.
Is there any way I can write either the EXE or the DLL such that the inline assembly instruction is not needed? For example, printf is also a __cdecl function yet I do not need to write inline assembly to ensure the stack is cleaned up after every call I make to it.
You need to make sure the calling convention of addVecs is correct. It pays to be explicit about this, and not rely on your compiler's defaults.
typedef VECTOR (__cdecl *addVecs_ptr)( VECTOR, VECTOR, VECTOR, VECTOR );
static addVecs_ptr addVecs = NULL;
If you've used a typedef like this, it makes the cast easy later on when you're loading the address:
addVecs = (addVecs_ptr) GetProcAddress( hSomeDllHandle, "addVecs" );