Search code examples
windowslinuxcalling-conventionavx

How are AVX registers handled by the common calling conventions?

I could not find any document defining how YMM registers are handled by the caller and by the callee.

To focus my question, here is what I would like to know:

  1. Which YMM registers must be restored by the callee before returning to the caller?
  2. Since there are differences for XMM registers in Linux and Windows, i assume that also YMM registers do not follow the same rules. what are the rules for each of the OSes?

EDIT: Thanks to the answer below I was able to scavenge the answer in the mentioned doc for Win64. I am quite sure that Linux follows similar rules:

"...
The YMM registers do not have callee-save status, except for the lower half
of YMM6-YMM15 in 64-bit Windows, where XMM6-XMM15 have callee-save status.
Possible future extensions of the vector registers to 512 bits or more will not have calleesave
status.
..."
Solution

  • The answer to both of your questions (assuming that you're talking about calling convention in C++) is in the great optimization guide by Agner Fog:

    Calling conventions for different C++ compilers and operating systems.

    See section 6 (Register usage) on page 10. Also section 7.2 (Passing and returning SIMD types) on page 22 might be relevant.