The code all worked fine, but then I added directives to create unwinding data, and YASM started refusing to assemble my code.
The build settings are for x64, as proven by "-f x64" in the auto-generated command line: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\"vsyasm.exe -Xvc -f x64 -o "x64\Debug\\" -l "" -P "" --mapdir= "" -E "" --prefix "" --suffix "" -rnasm -pnasm -m amd64
Normally the -m switch isn't in there, but I added it for good measure. That didn't help.
Changing the argument of the -f switch to win64 doesn't help either: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\"vsyasm.exe -Xvc -f win64 -o "x64\Debug\\" -l "" -P "" --mapdir= "" -E "" --prefix "" --suffix "" -rnasm -pnasm -m amd64
In this code,
bits 64
global s24_fl32_sse2_win64
s24_fl32_sse2_win64:
proc_frame s24_fl32_sse2_win64
sub rsp, 8 + 4 * 16
[alloc_stack 8 + 4 * 16]
movdqa [rsp], xmm6
[save_xmm128 xmm6, 0]
[endprolog]
VSYASM complains about a redefinition of the function label, which probably means that it didn't recognize proc_frame as a directive. It also says error: unrecognized directive 'alloc_stack' and the same for save_xmm128. It doesn't complain about endprolog.
Putting the proc_frame in brackets makes the first error go away (the label re-definition), but the manual says to use it without brackets, and it doesn't solve the problem anyway.
How do I fix this? Preferably in a way that creates proper unwind data.
I think PROC_FRAME defines the label, so you don't need the s24_fl32_sse2_win64: line - also a few examples I've seen put global s24_fl32_sse2_win64 after the PROC_FRAME - I'm not sure whether this is required or not.
In addition - alloc_stack and save_xmm128 (without square brackets) are macros (they handle both the stack operation itself and the related unwind primitive) - the unwind primitive versions of these are [allocstack ....] and [savexmm128....].