Search code examples
assemblywinapix86nasmcommand-line-arguments

Win64 NASM: Segfault in CommandLineArgvW


Any attempt to call CommandLineToArgvW in NASM using standard Windows calling convention results in it always segfaulting. The equivalent of this program in C works perfectly fine

NASM source code:

extern GetCommandLineW
extern CommandLineToArgvW
extern LocalFree
extern printf

section .rodata
    argc_format: db '%llu', 0

section .bss
    argc: resq 1

section .text
global main

main:
    sub rsp, 32

    call GetCommandLineW

    mov rcx, rax
    lea rdx, [rel argc]
    call CommandLineToArgvW;

    mov rcx, rax
    call LocalFree

    lea rcx, [rel argc_format]
    mov rdx, [rel argc]
    call printf

    add rsp, 32

    xor rax, rax
    ret

Result of it running:

...\main_nasm>main_nasm

...\main_nasm>echo %ERRORLEVEL%
-1073741819

GDB runthrough:

Reading symbols from main_nasm...
(gdb) break main
Breakpoint 1 at 0x140001540
(gdb) run
Starting program: ...\main_nasm.exe
[New Thread 10868.0x4ed0]
[New Thread 10868.0x3adc]
[New Thread 10868.0x479c]

Thread 1 hit Breakpoint 1, 0x00007ff6a6b71540 in main ()
(gdb) next
Single stepping until exit from function main,
which has no line number information.
0x00007ff6a6b7157c in GetCommandLineW ()
(gdb) next
Single stepping until exit from function GetCommandLineW,
which has no line number information.
0x00007ffac901f6d0 in KERNEL32!GetCommandLineW () from C:\Windows\System32\kernel32.dll
(gdb) next
Single stepping until exit from function KERNEL32!GetCommandLineW,
which has no line number information.
0x00007ffac8d00fd0 in KERNELBASE!GetCommandLineW () from C:\Windows\System32\KernelBase.dll
(gdb) next
Single stepping until exit from function KERNELBASE!GetCommandLineW,
which has no line number information.
0x00007ff6a6b71549 in main ()
(gdb) next
Single stepping until exit from function main,
which has no line number information.

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffac9da4220 in StrStrW () from C:\Windows\System32\shell32.dll
(gdb) next
Single stepping until exit from function StrStrW,
which has no line number information.

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffac9da4220 in StrStrW () from C:\Windows\System32\shell32.dll
(gdb) next
Single stepping until exit from function StrStrW,
which has no line number information.
[Thread 10868.0x479c exited with code 3221225477]
[Thread 10868.0x4ed0 exited with code 3221225477]
[Thread 10868.0x3adc exited with code 3221225477]

Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.

C source code:

#include <stdio.h>
#include <Windows.h>

int main()
{
    int argc = 0;
    LocalFree(CommandLineToArgvW(GetCommandLineW(), &argc));

    printf("%llu", argc);

    return 0;
}

Result of it running:

...\main_c>main_c
1
...\main_c>echo %ERRORLEVEL%
0

To be frank, I'm pretty new to NASM and my understanding of calling conventions might very well be the fault, but the tutorial I followed defined it the way I used it (allocate 32 bytes on stack, RCX - first arg, RDX - second, R8 - third, R9 - fourth, everything else - on stack with offset of 32, return result - RAX). Besides, works for GetCommandLineW and printf

I tried to implement it the

call GetCommandLineW

lea rbx, [rel argc]
push rbx
push rax
call CommandLineToArgvW

...way, but that doesn't change anything


Solution

  • Yes. The problem was, in fact, because my stack wasn't aligned to 16 bytes, as RbMm and Peter Cordes noted. I just completely forgot that a program's entry point also has its return address on the top of the stack, which led allocating 32 bytes being insufficient. Changing sub rsp, 32 and add rsp, 32 to sub rsp, 40 and add rsp, 40 fixed it.