Search code examples
c++jitcalling-convention

How do I know what registers function parameters will be mapped on?


I have written a JITting VM for Brainfuck. It requires the creation of a a writable+executable memory area (that it, a char*, but not created with malloc or new) that is filled with machine opcodes (x64 in my case), then called after converting the char* into a function pointer. That newly created function has parameters. Namely, the memory pointer and some other internal things.

It works, but it assumes a certain register mapping. That mapping has changed, once some time ago, because I changed the code around the call into the char* and the compiler decided it would work better with other registers. So I had to change the code generation scheme.

I'm afraid I could do something that would make the compiler change it again, or worse, use the stack, which I don't know how to pull parameters from (yet).

So I need:

  • a way to tell the compiler on what registers to put the parameters (`register ... asm("r0"), as described here, does not work with parameters)
  • else a way to know what registers the parameters were mapped on. That would complexify the thing, but it's still feasible.
  • else a way to know if and in what order the parameters were passed on the stack

I'm using gcc 4.8.2.


Solution

  • When you call a function, the compiler doesn't normally* have a choice about where to put the arguments for the call. This is dictated by the ABI. If your host program is written in C++ and runs on a Linux/x86-64 machine, it will generally emit code that conforms to the calling convention described in the SysV ABI, which contains the following rules for parameter passing:

    If the class is INTEGER, the next available register of the sequence %rdi, %rsi, %rdx, %rcx, %r8 and %r9 is used.

    It's worth noting that the INTEGER class includes pointer arguments as well.

    The bottom line here is that your jitted code ought to be able to pick its arguments out of the registers listed above. If this still fails to work I suspect you may be encoding your instructions incorrectly. Try viewing the disassembly from GDB.

    (*) Well, the compiler may be afforded some more flexibility if the function cannot be called from outside the translation unit.