Search code examples
assemblysegmentation-faultmasm32

SIGSEGV, Segmentation fault while doing "ret"


I'm learning how to work with procedures in masm32 so I wrote procedure that write a number:

.386
.model flat, stdcall
option casemap : none

include \masm32\include\masm32.inc
include \masm32\include\kernel32.inc
include \masm32\macros\macros.asm
includelib \masm32\lib\masm32.lib
includelib \masm32\lib\kernel32.lib

.data
    number dw 397
    temp db 10
    symbol dw ?
    i dw ?

.code
    printnumber proc num:WORD
    mov ecx, 0
    mov ax, num
    @@:
    mov edx, 0
    div temp
    mov bh, 0
    mov bl, ah    
    push bx
    inc cx
    cmp al, 0
    mov bl, al
    mov ax, bx
    jnz @B
    mov i, cx
    @@:
    pop symbol
    add symbol, 48
    mov ax, symbol
    print ADDR symbol
    dec i
    cmp i, 0
    jnz @B
    ret
printnumber endp

start:
    push number
    call printnumber
    ret ;here program fails
end start

Program prints "397" successfully, but after trying to do "ret" there is a problem: "Program received signal SIGSEGV, Segmentation fault.". What should I do?


Solution

  • You are unbalancing the stack by failing to properly clean it.

    At the top of your code file, you have this directive:

    .model flat, stdcall
    

    The important part is the stdcall directive, which specifies the calling convention that your functions will use. This is the most common calling convention for Windows programs, and it is the same one used by the Windows API. There are two important features of the stdcall calling convention:

    1. Arguments are pushed onto the stack from right-to-left.
    2. The callee is responsible to clean up the stack before it returns.

    The first feature is the same as the other common calling convention, cdecl, but the second is exactly opposite. It is what you are getting wrong in this case. (Actually, your code does not clean up the stack at all, so it would be broken regardless of calling convention!)

    Basically, when you're using the stdcall calling convention, you will use the version of the ret instruction that takes an immediate as an argument. That argument specifies the number of bytes to pop from the stack upon return.

    In this case, you have one argument, the WORD-sized number, so you would use ret 2:

    printnumber proc num:WORD
        mov ecx, 0
        mov ax, num
        @@:
        mov edx, 0
        div temp
        mov bh, 0
        mov bl, ah    
        push bx
        inc cx
        cmp al, 0
        mov bl, al
        mov ax, bx
        jnz @B
        mov i, cx
        @@:
        pop symbol
        add symbol, 48
        mov ax, symbol
        print ADDR symbol
        dec i
        cmp i, 0
        jnz @B
        ret 2         ; clean up the stack by popping 2 bytes,
                      ;  since 2 bytes were pushed by the caller
    printnumber endp
    

    Aside from that, I believe that with MASM32's start entry-point label, you have to call ExitProcess to return control to Windows. All the MASM32 samples I've seen do it this way:

    start:
        push number
        call printnumber
        invoke ExitProcess, 0
    end start
    

    But I could be wrong about this. I don't actually use the MASM32 SDK. Normally, your entry point would be a cdecl function that would simply return control with ret.