I got confused when reading about blocking send and receives in MPI. As stated in the MPI Standard 2.2, section 3.4 Communication Modes:
... The receive operation described in the last section is blocking: it returns only after the receive buffer contains the newly received message. A receive can complete before the matching send has completed (of course, it can complete only after the matching send has started).
Can somebody explain to me how a blocking receive can complete before the corresponding send has completed? My understanding is that a blocking receive (MPI_recv) returns when data is ready to be used in the receive buffer (i.e. the data has been received fully). Is this not the case?
Your understanding is correct.
The described case (receive completes before send completes) in the standard is telling you not to rely on the ordering of the library calls. It does not necessary need to be the case on a specific machine / implementation.
If you think about a synchronous send and a blocking receive of a tiny message. The synchronous send may only complete after the matching receive has started, possibly indicated by 'confirmation message (ACK)'. The actual message transfer might completely transferred (receive may return) before the ACK arrives at the sender (send may return). Please keep in mind this is highly implementation / machine specific.