Before I proceed, I'd like to mention that I did try to research this topic on the internet, but I still need clarification.
Let's say I have two Linux machines connected to a switch (and only to a switch). Machine A has an IP address of 10.0.0.1 and machine B -- 10.0.0.2. I used nmcli command to set the IP address and create an ethernet interface for each machine. Everything works as expected.
Now, the confusing part is how machine A can find machine B and vice versa? I'm using the following command to connect from machine A to machine B:
And it works, even if this is the very first data transmission. This surely means that machine A somehow already knew the machine's B MAC address; otherwise, the frame wouldn't find its way to machine B. But how? Since the IP address is meaningless to the switch (Level2), why when I do ping 10.0.0.2 or ssh 10.0.0.2, it still works?
Probably the ARP cache was already populated. Maybe there was a grations ARP broadcast:
Every time an IP interface or link goes up, the driver for that interface will typically send a gratuitous ARP to preload the ARP tables of all other local hosts.
If not, most likely an ARP request/reply was happening right before the first ping. Check the arp command or ip neigh.
In general I suggest you use Wireshark to explore what's going on, or something like tcpdump -n -i eth0 not ssh if your are working remotely (note the -n to prevent name resolution). You can also record traffic with tcpdump -s 9999 -w output.pcap and view it later in Wireshark.
If you sniff network traffic on a third PC, keep in mind that switches will not send traffic to all ports when they have learned where the destination is. Some switches allow you to configure a mirror port to observe all traffic to or from a certain port. Either way you should always be able to observe ARP requests as they are broadcast.