Any attempt to call CommandLineToArgvW in NASM using standard Windows calling convention results in it always segfaulting. The equivalent of this program in C works perfectly fine
NASM source code:
extern GetCommandLineW
extern CommandLineToArgvW
extern LocalFree
extern printf
section .rodata
argc_format: db '%llu', 0
section .bss
argc: resq 1
section .text
global main
main:
sub rsp, 32
call GetCommandLineW
mov rcx, rax
lea rdx, [rel argc]
call CommandLineToArgvW;
mov rcx, rax
call LocalFree
lea rcx, [rel argc_format]
mov rdx, [rel argc]
call printf
add rsp, 32
xor rax, rax
ret
Result of it running:
...\main_nasm>main_nasm
...\main_nasm>echo %ERRORLEVEL%
-1073741819
GDB runthrough:
Reading symbols from main_nasm...
(gdb) break main
Breakpoint 1 at 0x140001540
(gdb) run
Starting program: ...\main_nasm.exe
[New Thread 10868.0x4ed0]
[New Thread 10868.0x3adc]
[New Thread 10868.0x479c]
Thread 1 hit Breakpoint 1, 0x00007ff6a6b71540 in main ()
(gdb) next
Single stepping until exit from function main,
which has no line number information.
0x00007ff6a6b7157c in GetCommandLineW ()
(gdb) next
Single stepping until exit from function GetCommandLineW,
which has no line number information.
0x00007ffac901f6d0 in KERNEL32!GetCommandLineW () from C:\Windows\System32\kernel32.dll
(gdb) next
Single stepping until exit from function KERNEL32!GetCommandLineW,
which has no line number information.
0x00007ffac8d00fd0 in KERNELBASE!GetCommandLineW () from C:\Windows\System32\KernelBase.dll
(gdb) next
Single stepping until exit from function KERNELBASE!GetCommandLineW,
which has no line number information.
0x00007ff6a6b71549 in main ()
(gdb) next
Single stepping until exit from function main,
which has no line number information.
Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffac9da4220 in StrStrW () from C:\Windows\System32\shell32.dll
(gdb) next
Single stepping until exit from function StrStrW,
which has no line number information.
Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffac9da4220 in StrStrW () from C:\Windows\System32\shell32.dll
(gdb) next
Single stepping until exit from function StrStrW,
which has no line number information.
[Thread 10868.0x479c exited with code 3221225477]
[Thread 10868.0x4ed0 exited with code 3221225477]
[Thread 10868.0x3adc exited with code 3221225477]
Program terminated with signal SIGSEGV, Segmentation fault.
The program no longer exists.
C source code:
#include <stdio.h>
#include <Windows.h>
int main()
{
int argc = 0;
LocalFree(CommandLineToArgvW(GetCommandLineW(), &argc));
printf("%llu", argc);
return 0;
}
Result of it running:
...\main_c>main_c
1
...\main_c>echo %ERRORLEVEL%
0
To be frank, I'm pretty new to NASM and my understanding of calling conventions might very well be the fault, but the tutorial I followed defined it the way I used it (allocate 32 bytes on stack, RCX - first arg, RDX - second, R8 - third, R9 - fourth, everything else - on stack with offset of 32, return result - RAX). Besides, works for GetCommandLineW and printf
I tried to implement it the
call GetCommandLineW
lea rbx, [rel argc]
push rbx
push rax
call CommandLineToArgvW
...way, but that doesn't change anything
Yes. The problem was, in fact, because my stack wasn't aligned to 16 bytes, as RbMm and Peter Cordes noted. I just completely forgot that a program's entry point also has its return address on the top of the stack, which led allocating 32 bytes being insufficient. Changing sub rsp, 32
and add rsp, 32
to sub rsp, 40
and add rsp, 40
fixed it.