Search code examples
linuxassemblyx86nasm

Nasm segmentation fault on RET in _start


section .text
     global _start
_start:
     nop
main:
     mov eax, 1
     mov ebx, 2
     xor eax, eax
     ret

I compile with these commands:

nasm -f elf main.asm
ld -melf_i386 -o main main.o

When I run the code, Linux throw a segmentation fault error

(I am using Linux Mint Nadia 64 bits). Why this error is produced?


Solution

  • Because ret is NOT the proper way to exit a program in Linux, Windows, or Mac!!!!

    _start is not a function, there is no return address on the stack because there is no user-space caller to return to. Execution in user-space started here (in a static executable), at the process entry point. (Or with dynamic linking, it jumped here after the dynamic linker finished, but same result).

    On Linux / OS X, the stack pointer is pointing at argc on entry to _start (see the i386 or x86-64 System V ABI doc for more details on the process startup environment); the kernel puts command line args into user-space stack memory before starting user-space. (So if you do try to ret, EIP/RIP = argc = a small integer, not a valid address. If your debugger shows a fault at address 0x00000001 or something, that's why.)


    For Windows it is ExitProcess and Linux is is system call - int 80H using sys_exit, for x86 or using syscall using 60 for 64-bit or a call to exit from the C Library if you are linking to it.

    32-bit Linux (i386)

    %define  SYS_exit  1   ; call number __NR_exit from <asm/unistd_32.h>
    
    mov     eax, SYS_exit  ; use the NASM macro we defined earlier
    xor     ebx, ebx       ; ebx = 0  exit status
    int     80H            ; _exit(0)
    

    64-bit Linux (amd64)

    mov     rax, 60        ; SYS_exit aka __NR_exit from asm/unistd_64.h
    xor     rdi, rdi       ; edi = 0  first arg to 64-bit system calls
    syscall                ; _exit(0)
    

    (In GAS you can actually #include <sys/syscall.h> or <asm/unistd.h> to get the right numbers for the mode you're assembling a .S for, but NASM can't easily use the C preprocessor. See Polygot include file for nasm/yasm and C for hints.)

    32-bit Windows (x86)

    push    0
    call    ExitProcess
    

    Or Windows/Linux linking against the C Library

    ; pass an int exit_status as appropriate for the calling convention
    ; push 0   /  xor edi,edi  /  xor ecx,ecx
    call    exit
    

    (Or for 32-bit x86 Windows, call _exit, because C names get prepended with an underscore, unlike in x86-64 Windows. The POSIX _exit function would be call __exit, if Windows had one.)

    Windows x64's calling convention includes shadow space which the caller has to reserve, but exit isn't going to return so it's ok to let it step on that space above its return address. Also, 16-byte stack alignment is required by the calling convention before call exit except for 32-bit Windows, but often won't actually crash for a simple function like exit().


    call exit (unlike a raw exit system call or libc _exit) will flush stdio buffers first. If you used printf from _start, use exit to make sure all output is printed before you exit, even if stdout is redirected to a file (making stdout full-buffered, not line-buffered).

    It's generally recommended that if you use libc functions, you write a main function and link with gcc so it's called by the normal CRT start functions which you can ret to.

    See also

    Defining main as something that _start falls through into doesn't make it special, it's just confusing to use a main label if it's not like a C main function called by a _start that's prepared to exit after main returns.