NASM call label doesn't get called

Question

I'm doing some assembly coding using NASM on Linux but I've run into a problem.

global _start

section .text

testFunctionName:
    mov rdi, 12
    ret

_start:

    call testFunctionName

    mov rax, 1
    mov rbx, 0
    int 0x80

then assembled and linked:

nasm -felf64 -F dwarf -g program.asm -o program.o
ld program.o -o program

testFunctionName is never jumped to. I verified this using GDB. I haven't found relatable explanations elsewhere. If somebody could explain what's happening and how to fix this, I would appreciate it.

Answer 1

You're probably using nexti or next which steps over calls, not into them.

Set a breakpoint on the mov rdi, 12 inside your function, or single-step into calls using stepi (or si for short).

Or if you don't trust GDB, write a program whose exit status depends on whether the function runs or not.

testFunctionName:
    mov  ebx, 12
    ret

_start:
    xor   ebx, ebx   ; ebx=0

    call testFunctionName

    mov   eax, 1
    int   0x80       ; sys_exit(ebx), 32-bit ABI

Run this and echo $? to see that the exit status is 12, not 0, because the function does run unless your CPU is broken, or NASM or your linker are broken.

---

You could also put an illegal instruction inside your test function, like ud2. Or a software breakpoint: int3. Or a divide by zero. Something that will raise an exception instead of letting your process make its exit system call. Then you'll definitely notice whether it's called or not.

---

See also What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code? To make a native 64-bit exit(0) system call the same way glibc's _exit() function does, do

    xor   edi, edi     ; rdi=0
    mov   eax, 231     ; __NR_exit_group   from <asm/unistd_64.h>
    syscall            ; sys_exit_group(edi)