If I have a Memory-mapped I/O device, and I want to write to a register for this device located at address 0x16D34, the 0x16D34 address is actually a virtual address, and the CPU will translate it to a physical address first, and then write the data to the physical address.
But what about Port-mapped I/O devices (for example: a serial port), so if I want to write to a register for a serial port located at address 0x3F8, is the 0x3F8 address a physical address or a virtual address?
Edit: I am on x86 architecture.
Port-mapped I/O on x86/x86-64 (most other modern architectures don't even support it) happens in an entirely separate address space. This address space is not subject to memory mapping, so there are no virtual port addresses, only physical ones. Special in and out instructions must be used to perform port I/O, simple memory access (e.g. with mov) can't access this separate address space. Access protection based on privilege level is possible; most modern OSes prevent user space processes from accessing I/O ports by default.
For details, you can for example check the chapter "INPUT/OUTPUT" of Intel's "Intel® 64 and IA-32 Architectures Developer's Manual: Vol. 1" (chapter 18 as of this writing).
Note that in the early days of x86, port addresses were hardwired in each device, including ISA add-in cards. If you were lucky, the card had a set of jumpers for selecting one of a limited set of possible port ranges for the device, in order to avoid range clashes between devices. Later, Plug & Play was introduced to make the selection dynamically during system boot. PCI further refined this, so that I/O BARs can pretty much be mapped anywhere within the 0x0000-0xffff address space by the operating system and/or firmware. Port-mapped I/O is now strongly discouraged when designing new hardware due to its many inherent limitations.