I can't wrap my head around how packet-sniffers can be used by anyone on the network.
I know very little about how networks work, but let me put it this way: suppose the mailman comes around delivering a package to my doorstep. Why is is that I'm able to rifle though all his other packages and look around? Shouldn't the mailman only hand me packages that are mine?
To quote William Pursell's comment, which he should have made an answer and should have expanded:
The mailman does not deliver the letter your doorstep. Instead, he opens your mail and shouts out: "this letter is for <name>. No-one else should listen" and then proceeds to read the letter out loud. –
In the original Ethernet network, there was a shared cable to which all hosts were attached; if a host wanted to send a message to another host, it would transmit the packet on the shared cable, with an Ethernet header with the destination Ethernet address of the other host. All hosts on the cable could, in theory, see the packet. (This was in an era where security was less of a concern; for cases where security was a concern, the packets were encrypted in a fashion that the other host could fairly easily decrypt but that other hosts would have to decrypt in some other more difficult fashion.)
In addition, a packet can be sent to the "broadcast" Ethernet address (all 1's) or a "multicast" Ethernet address (which several hosts are configured to handle); broadcast packets are intended for all hosts on the Ethernet to see, and multicast packets are intended for all hosts in the address's "multicast group" to see.
Normally, an Ethernet adapter would ignore packets that aren't sent to its Ethernet address, to the broadcast Ethernet address, or to a multicast address for which it's configured to receive packets. Most can, however, be put into "promiscuous" mode, where they pass all packets to the host; that mode is used for packet sniffers.
Most current Ethernets are "switched"; instead of a shared cable, there's an Ethernet switch, and hosts plug into the switch with a cable. Packets sent to a particular host's Ethernet address will only be sent out the switch port for that host (unless somebody's configured the hosts to have a "mirror port" on which all traffic is sent, or unless the switch hasn't yet determined which port is the port for that Ethernet address). Broadcast packets are sent to all ports, and multicast packets may be sent to all ports or, if the switch can determine that, to those ports that have adapters configured for the multicast address in question.
Wi-Fi networks are similar, but they're usually protected with encryption, as it's easier for somebody to bring in a laptop and put it into "monitor mode" to sniff on a given channel than it is for somebody to bring in a laptop, configure a switch to have a mirror port (or use some other mechanism to get access to the traffic), and plug the laptop into the appropriate port on the switch.