Search code examples
cgccx86-64inline-assemblyred-zone

How do I tell gcc that my inline assembly clobbers part of the stack?


Consider inline assembly like this:

uint64_t flags;
asm ("pushf\n\tpop %0" : "=rm"(flags) : : /* ??? */);

Nonwithstanding the fact that there is probably some kind of intrinsic to get the contents of RFLAGS, how do I indicate to the compiler that my inline assembly clobbers one quadword of memory at the top of stack?


Solution

  • Apart from Peter Cordes's approach of skipping the redzone:

    long getflags0(void){
        long f; __asm(
            "add $-128, %%rsp;\n"
            "pushf; pop %0;\n"
            "sub $-128, %%rsp\n" : "=r"(f) :: );
        return f;
    }
    

    which renders:

    0000000000000000 <getflags0>:
       0:   48 83 c4 80             add    $0xffffffffffffff80,%rsp
       4:   9c                      pushfq 
       5:   58                      pop    %rax
       6:   48 83 ec 80             sub    $0xffffffffffffff80,%rsp
       a:   c3                      retq   
    $sz(getflags0)=11
    

    you can also just list rsp as a clobber and silence the deprecation warning:

    long getflags(void){
        long f;
        #pragma GCC diagnostic push
        #pragma GCC diagnostic ignored "-Wdeprecated"
        __asm("pushf; pop %0" : "=r"(f) :: "rsp");
        #pragma GCC diagnostic pop
        return f;
    }
    

    which renders:

    000000000000000b <getflags>:
       b:   55                      push   %rbp
       c:   48 89 e5                mov    %rsp,%rbp
       f:   9c                      pushfq 
      10:   58                      pop    %rax
      11:   c9                      leaveq 
      12:   c3                      retq   
    $sz(getflags)=8
    

    From experience (played with this quite a bit), gcc actually handles rsp clobbers quite well -- by forcing a frame pointer (which it won't let you clobber alongside rsp -- that's a hard assembler error), avoiding the redzone, addressing locals relatively to the frame pointer, and by forcing an %rsp restoring code at the end of the function.

    The mechanism of making the compiler let go of the end of the stack is needed for VLAs and allocas anyway, so I don't think it's going anywhere.

    I think such rsp clobbers are quite usable for custom stack allocations, frees, and stack switches, as long as you don't mess with what the compiler spilled below the stack pointer it gave you (or open it up to being messed with).

    I only had some issues with this approach on clang, but the fix to the compiler seems trivial: https://github.com/llvm/llvm-project/issues/61898.

    As for suppressing the warnings without affecting the whole compilation unit,

    #pragma GCC diagnostic push
    #pragma GCC diagnostic ignored "-Wdeprecated"
    //...
    #pragma GCC diagnostic pop
    

    can work well inside a (possibly inline -- had no issues with rsp clobbers inside inline functions either) function, or you can generate the pragma with _Pragma to make it usable inside of macros.

    Clang doesn't complain about rsp clobbers (though you will run into issues on it if you use rsp clobbers on it for memory allocation unless you apply my fix to a custom build) unless you compile with -fstack-clash-protection. Then the warning is -Wstack-protector, and it's silenceable equivalently.

    Please keep in mind that while this happens to work, it is not officially supported. From https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#Clobbers-and-Scratch-Registers-1:

    The compiler requires the value of the stack pointer to be the same after an asm statement as it was on entry to the statement. However, previous versions of GCC did not enforce this rule and allowed the stack pointer to appear in the list, with unclear semantics. This behavior is deprecated and listing the stack pointer may become an error in future versions of GCC.