I am learning amd64 assembler, and trying to implement a simple Unix filter. For an unknown reason, even simplified to the bare minimum version (code below), it crashes at random.
I tried to debug this program in GNU Debugger (gdb). In the default configuration of gdb, the program runs fine, but if I enable address randomization (set disable-randomization off), the program starts crashing (SIGSEGV). The problematic instruction is marked in the listing:
format ELF64 executable
sys_read = 0
sys_write = 1
sys_exit = 60
entry $
foo:
label .inbuf at rbp - 65536
label .outbuf at .inbuf - 65536
label .endvars at .outbuf
mov rbp, rsp
mov rax, sys_read
mov rdi, 0
lea rsi, [.inbuf]
mov rdx, 65536
syscall
xor ebx, ebx
cmp eax, ebx
jl .read_error
jz .exit
mov r8, rax ; r8 - count of valid bytes in input buffer
xor r9, r9 ; r9 - index of byte in input buffer, that is being processed.
xor r10, r10 ; r10 - index of next free position in output buffer.
.next_byte:
cmp r9, r8
jg .exit
mov al, [.inbuf + r9]
mov [.outbuf + r10], al ;; SIGSEGV here in GDB
inc r10
inc r9
jmp .next_byte
.read_error:
mov rax, sys_exit
mov rdi, 1
syscall
.exit:
mov rax, sys_write
mov rdi, 1
lea rsi, [.outbuf]
mov rdx, r10
syscall
mov rax, sys_exit
xor rdi, rdi
syscall
This program is meant to read at most 64kB from stdin, store it into a buffer on the stack, copy the read data byte-by-byte into the output buffer and write the content of the output buffer to the standard output stream. Essentially, it should behave as a limited version of cat.
On my computer, it either works as intended, or crashes with SIGSEGV, with an approximate rate of 1 successful run to 4 crashes.
The red zone in amd64 is only 128 bytes long, but you're using 131072 bytes below rsp. Move the stack pointer down to encompass the buffers that you want to store on the stack.