pointer casting in c [shellcode test]

Question

I'm following a tutorial on how to write a shellcode but I'm failing to understand what this pointer function casting is doing to the bytecode, could someone explain this to me?.

char code[] = "bytecode will go here!";
int main(int argc, char **argv)
{
   int (*func)();
   func = (int (*)()) code;
   (int)(*func)();
}

the code is something like this "\xb0\x01\x31\xdb\xcd\x80" for exiting it's the result of objdump of an assembly code that exits

Answer 1

The code itself has not only undefined behaviour, it is bad wherever it doesn't. Let's go through the lines:

1. Declare func as a pointer to a function returning an int, with no prototype.

   int (*func)();
   

2. cast the array which decays to a pointer to the first character in the array, into a pointer to function returning int having no prototype, and assign it to func. This of course has totally undefined behaviour.

   func = (int (*)()) code;
   

3. The third line is the worst of them all:

   (int)(*func)();
   

   Here we dereference the function pointer with *func, then it immediately decays back to a function pointer because it is subjected to the function call operator which only works with function pointers! Then the return value, which is an int, is cast to an int, and then discarded.

Of course the 3rd line should have been written as func();. And since the return value is very probably handled via registers, and we're not interested in it, the proto for the shellcode function should have been void shellcode(void). Lastly, you don't need a variable here for the function pointer. Therefore we could just write

( (void (*)(void)) code ) ();

i.e. cast the code as a pointer to function returning void taking no arguments, then call it.