I was experimenting with the NASM assembler, when I came across a problem:
mov (sp),bx
mov [sp],bx
The first instruction is assembled properly while the second one is not, and gives me the error:
error: invalid effective address
Why is this? What's the difference between the two?
(%sp) would be an AT&T syntax addressing mode. (Invalid because 16-bit addressing modes can't use SP directly, only BP|BX + SI|DI NASM x86 16-bit addressing modes; that's also the reason mov [sp], bx is invalid.)
In NASM syntax, square brackets [] mean a memory operand.
In NASM, the parens () around SP are removed just like any compile-time expression,
so mov (sp), bx assembles to 89DC mov sp,bx. Try it yourself by assembling and using ndisasm on the output. (Or assemble into -felf32 and use objdump)
This is a mov between two registers, overwriting the stack pointer. Very likely not what you want, and totally different from storing to memory with mov [bp], bx or whatever.
In NASM, you might use parens when writing something like mov ax, (1+3) * 4 so NASM's expression parser handles parens, and apparently having a register name inside parens doesn't change anything.
I only mentioned AT&T syntax at the top of this answer because that and Plan9/Go syntax are the only time you'd normally put a register name inside parens; it's just confusing in NASM syntax; don't do it.