Search code examples
assemblygccx86inline-assemblyregister-allocation

Concrete example of incorrect behavior of an early-clobber affecting a memory operand's addressing mode in GCC inline asm?


Below is excerpted from the GCC manual's Extended Asm docs, on embedding assembly instructions in C using asm keyword:

The same problem can occur if one output parameter (a) allows a register constraint and another output parameter (b) allows a memory constraint. The code generated by GCC to access the memory address in b can contain registers which might be shared by a, and GCC considers those registers to be inputs to the asm. As above, GCC assumes that such input registers are consumed before any outputs are written. This assumption may result in incorrect behavior if the asm statement writes to a before using b. Combining the ‘&’ modifier with the register constraint on a ensures that modifying a does not affect the address referenced by b. Otherwise, the location of b is undefined if a is modified before using b.

The italic sentence says there may be "incorrect behavior" if the asm statement writes to a before using b.

I cannot figure out how such an "incorrect behavior" could have occurred, so I wish to have a concrete asm code example to demonstrate the "incorrect behavior" so that I could have a deep understanding of this paragraph.

I can perceive the problem when two such asm codes are running in parallel, but the above paragraph does not mention multiprocessing scenario.

If we have only one CPU with one core, can you please show an asm code that may produce such an incorrect behavior, that is, modifying a affects the address referenced by b such that the location of b is undefined.

The only assembly language I am familiar with is Intel x86 assembly, so please make the example targeted on that platform.


Solution

  • Consider the following example:

    extern int* foo();
    int bar()
    {
        int r;
    
        __asm__(
            "mov $0, %0 \n\t"
            "add %1, %0"
        : "=r" (r) : "m" (*foo()));
    
        return r;
    }
    

    The usual calling convention puts return values into the eax register. As such, there is a good chance the compiler decides to use eax throughout, to avoid unnecessary copying. The generated assembly may look like:

            subl    $12, %esp
            call    foo
            mov $0, %eax
            add (%eax), %eax
            addl    $12, %esp
            ret
    

    Notice that the mov $0, %eax zeroes eax before the next instruction attempts to use it for referencing the input argument, hence this code will crash. With early clobber, you force the compiler to pick different registers. In my case, the resulting code was:

            subl    $12, %esp
            call    foo
            mov $0, %edx
            add (%eax), %edx
            addl    $12, %esp
            movl    %edx, %eax
            ret
    

    The compiler could have instead moved the result of foo() into edx (or any other free register), like this:

            subl    $12, %esp
            call    foo
            mov     %eax, %edx
            mov $0, %eax
            add (%edx), %eax
            addl    $12, %esp
            ret
    

    This example used the memory constraint for an input argument, but the concept applies equally to outputs too.