Search code examples
cassemblyx86att

Segmentation fault when calling x86 Assembly function from C program


I am writing a C program that calls an x86 Assembly function which adds two numbers. Below are the contents of my C program (CallAssemblyFromC.c):

#include <stdio.h>
#include <stdlib.h>

int addition(int a, int b);

int main(void) {
    int sum = addition(3, 4);
    printf("%d", sum);
    return EXIT_SUCCESS;
}

Below is the code of the Assembly function (my idea is to code from scratch the stack frame prologue and epilogue, I have added comments to explain the logic of my code) (addition.s):

.text

# Here, we define a function addition
.global addition
addition:
    # Prologue:
    # Push the current EBP (base pointer) to the stack, so that we
    # can reset the EBP to its original state after the function's
    # execution
    push %ebp
    # Move the EBP (base pointer) to the current position of the ESP
    # register
    movl %esp, %ebp

    # Read in the parameters of the addition function
    # addition(a, b)
    #
    # Since we are pushing to the stack, we need to obtain the parameters
    # in reverse order:
    # EBP (return address) | EBP + 4 (return value) | EBP + 8 (b) | EBP + 4 (a)
    #
    # Utilize advanced indexing in order to obtain the parameters, and
    # store them in the CPU's registers
    movzbl 8(%ebp), %ebx
    movzbl 12(%ebp), %ecx

    # Clear the EAX register to store the sum
    xorl %eax, %eax
    # Add the values into the section of memory storing the return value
    addl %ebx, %eax
    addl %ecx, %eax

I am getting a segmentation fault error, which seems strange considering that I think I am allocating memory in accordance with the x86 calling conventions (e.x. allocating the correct memory sections to the function's parameters). Furthermore, if any of you have a solution, it would be greatly appreciated if you could provide some advice as to how to debug an Assembly program embedded with C (I have been using the GDB debugger but it simply points to the line of the C program where the segmentation fault happens instead of the line in the Assembly program).


Solution

    1. Your function has no epilogue. You need to restore %ebp and pop the stack back to where it was, and then ret. If that's really missing from your code, then that explains your segfault: the CPU will go on executing whatever garbage happens to be after the end of your code in memory.

    2. You clobber (i.e. overwrite) the %ebx register which is supposed to be callee-saved. (You mention following the x86 calling conventions, but you seem to have missed that detail.) That would be the cause of your next segfault, after you fixed the first one. If you use %ebx, you need to save and restore it, e.g. with push %ebx after your prologue and pop %ebx before your epilogue. But in this case it is better to rewrite your code so as not to use it at all; see below.

    3. movzbl loads an 8-bit value from memory and zero-extends it into a 32-bit register. Here the parameters are int so they are already 32 bits, so plain movl is correct. As it stands your function would give incorrect results for any arguments which are negative or larger than 255.

    4. You're using an unnecessary number of registers. You could move the first operand for the addition directly into %eax rather than putting it into %ebx and adding it to zero. And on x86 it is not necessary to get both operands into registers before adding; arithmetic instructions have a mem, reg form where one operand can be loaded directly from memory. With this approach we don't need any registers other than %eax itself, and in particular we don't have to worry about %ebx anymore.

    I would write:

    .text
    
    # Here, we define a function addition
    .global addition
    addition:
        # Prologue:
        push %ebp
        movl %esp, %ebp
    
        # load first argument
        movl 8(%ebp), %eax 
        # add second argument
        addl 12(%ebp), %eax
    
        # epilogue
        movl %ebp, %esp  # redundant since we haven't touched esp, but will be needed in more complex functions 
        pop %ebp
        ret
    

    In fact, you don't need a stack frame for this function at all, though I understand if you want to include it for educational value. But if you omit it, the function can be reduced to

    .text
    .global addition
    addition:
        movl 4(%esp), %eax
        addl 8(%esp), %eax
        ret