Search code examples
cassemblyx86x86-64calling-convention

x86 Assembly subroutines - understanding how to return local var struct using only 1 register


I have searched online about best practice of Assembly calling convention, and indeed they say eax should return the value, or the address of the data structure passed in the subroutine by the calling program. Eax (or any register) should not return the local variable address as after the subroutine is done, the address is no longer valid. However, I wrote a simple C program to testify this.

#include <stdio.h>

typedef struct {
    int numerator;
    int denominator;
} Fraction;
    
Fraction add_fractions(Fraction *frac1, Fraction *frac2) 
{
    // Not using malloc here, so Fraction fraction data should be all inside the local variables. 
    Fraction fraction = { 
        frac1->numerator + frac2->numerator, 
        frac1->denominator + frac2->denominator 
    }; 
    // Of course not the right calculation of sum of fractions, 
    // but the main point is to look at how the fraction variable here is returned to the main.

    return fraction;
}

int main()
{
    Fraction frac1 = { 1, 2 };
    Fraction frac2 = { 1, 3 };
    
    Fraction result = add_fractions(&frac1, &frac2);    
    printf("%d, %d\n", result.numerator, result.denominator);
}

And the compiled ASM version of the C code having the "main" and "add_fraction" subroutines:

add_fractions:
    pushq   %rbp
    .seh_pushreg    %rbp
    movq    %rsp, %rbp
    .seh_setframe   %rbp, 0
    subq    $16, %rsp
    .seh_stackalloc 16
    .seh_endprologue
    movq    %rcx, 16(%rbp)
    movq    %rdx, 24(%rbp)
    movq    16(%rbp), %rax
    movl    (%rax), %edx
    movq    24(%rbp), %rax
    movl    (%rax), %eax
    addl    %edx, %eax
    movl    %eax, -8(%rbp)
    movq    16(%rbp), %rax
    movl    4(%rax), %edx
    movq    24(%rbp), %rax
    movl    4(%rax), %eax
    addl    %edx, %eax
    movl    %eax, -4(%rbp)
    movq    -8(%rbp), %rax
    addq    $16, %rsp
    popq    %rbp
    ret
main:
    pushq   %rbp
    .seh_pushreg    %rbp
    movq    %rsp, %rbp
    .seh_setframe   %rbp, 0
    subq    $64, %rsp
    .seh_stackalloc 64
    .seh_endprologue
    call    __main
    movl    $1, -8(%rbp)
    movl    $2, -4(%rbp)
    movl    $1, -16(%rbp)
    movl    $3, -12(%rbp)
    leaq    -16(%rbp), %rdx
    leaq    -8(%rbp), %rax
    movq    %rax, %rcx
    call    add_fractions
    movq    %rax, -24(%rbp)
    movl    -20(%rbp), %edx
    movl    -24(%rbp), %eax
    movl    %edx, %r8d
    movl    %eax, %edx
    leaq    .LC0(%rip), %rax
    movq    %rax, %rcx
    call    printf
    movl    $0, %eax
    addq    $64, %rsp
    popq    %rbp
    ret

I'm not familiar with the AT&T syntax, but as far as I can understand, the main func passed into the add_fractions addresses of 2 fractions through 2 registers: rdx and rcx.

  • The add_fractions preserves spaces for local vars subq $16, %rsp.
  • Do some adding, then movl %eax, -8(%rbp): store the sum into the local numerator.
  • Do some more adding, and movl %eax, -4(%rbp): store the sum into the local denominator.
  • After adding the values, before restoring the rsp value, movq -8(%rbp), %rax stores the address (or value, I'm not sure) of the first local variable to rax to return to main.

I think the returned rax should have the address of the local vars so that the main can access both calculated numerator and denominator. Otherwise, main only sees one value from rax. IF this is true, then is it against the recommendation I found online as after leaving the subroutine by addq $16, %rsp, the address of local variable is no longer valid.

I'm getting it wrong somewhere? Thank you for helping explain this.


Solution

  • Eax (or any register) should not return the local variable address as after the subroutine is done, the address is no longer valid. However, I wrote a simple C program to testify this.

    No, you did not test returning a local address. Your program returns the value of a structure,1 not the address of a structure.

    Your Fraction structure contains two four-byte members. At one point in the routine, movl %eax, -8(%rbp) put the four bytes of the first member in memory at the location -8(%rbp). Near the end of the routine, movl %eax, -4(%rbp) put the four bytes of the second member in memory at location -4(%rbp), which is immediately after the four bytes of the first member.

    Then movq -8(%rbp), %rax loads all eight of those bytes into the processor register %rax. The entire eight-byte structure is returned in the single processor register %rax.

    In the main routine, movq %rax, -24(%rbp) stores those eight bytes in memory at the location -24(%rbp). Then movl -20(%rbp), %edx and movl -24(%rbp), %eax load the individual members into separate processor registers.

    Footnote

    1 Mathematically, the value of a structure is an ordered tuple of the values of its members. It is represented by a sequence of bytes composed of the representations of its members, including padding bytes.