Search code examples
assemblyx86-64calling-conventionabireturn-by-value

how is a struct returned by value, in terms of assembly language, if that struct is too large to fit in a register?


In C++ you can return a user-defined type by value. In x86-64 ASM return by value is implemented by moving the return value to RAX and popping the stored previous value of RIP into RIP to return to the caller. How can a struct or really any type of array that is larger than RAX be returned by value?


Solution

  • The SYSV x86_64 calling conventions (used by everyone except Microsoft) allow structures of up to 16 bytes and INTEGER classification to be returned in the RAX/RDX register pair, while those of SSE classification and up to 32 bytes can be returned in the XMM0/XMM1 register pair.

    The classification of a struct depends on the types of the fields in the struct, but basically integer and pointer types will be INTEGER while float and double will be SSE.

    Larger structs will get MEMORY classification, so will require an extra hidden argument (passed in RDI, so prepended to the existing arguments) specifying a pointer to memory that the return value will be written to. This pointer will be returned in RAX.

    This is all detailed in the SYSV x86_64 ABI doc