I have the following mov instruction (without the suffix) from a disassembler.
mov %dx,(%eax)
What would be the instruction suffix? First I thought that the destination register determines the suffix, however according to the book I'm reading, I guess it's determined by the "smallest" register. So in this case would be
movw %dx, (%eax)
since %dx (16-bit word register) is the smallest one. Is my reasoning correct? (Sometimes the CSAPP book is a little bit confusing, doesn't explain details clearly).
The destination isn't a register in your examples, it's the source that's a register. So the operand-size is 16-bit, thus AT&T would use movw.
The destination is 2 bytes in memory, selected by a 32-bit addressing mode. mov requires both source and dest to be the same width. If at least one operand is a register, that uniquely determines the operand size.
You need an explicit suffix for something like mov $123, (%eax) because neither operand is a register.
Your idea of "smallest" is totally bogus. movl %eax, (%bx) is movl because the register operand is 32 bits, and the 4-byte destination is selected by a 16-bit addressing mode.
The register or registers in the addressing mode have zero effect on the operand-size. Address-size and operand-size are independent, and you can override one but not the other. (That's why there are separate machine-code prefix bytes for operand-size (0x66) and address-size (0x67).