I have been reading about UDP sockets in python.
By default a socket is configured so that sending or receiving data blocks, stopping program execution until the socket is ready. Calls to send() wait for buffer space to be available for the outgoing data, and calls to recv() wait for the other program to send data that can be read.
I understand the receiving part, it has to wait till the other end sends. but why would the send block? It says in the book: "send() waits for buffer space to be available". What's the buffer space?
Is it for the whole Operating System or is it defined for every application running.
Is there a way to know how much buffer space is available?
The buffer referred to by the BSD sockets and POSIX documentation, and presumably also by whatever you were reading, is a per-socket buffer, not per-system or per-app. It's the same one you can set with SO_SNDBUF. When that buffer is full, send will block.
But the real question is, why would that buffer get full?
Underneath the stuff your app can see, the kernel usually has something like a ring of transmit buffers per NIC. When the NIC finishes putting a buffer worth of data out on the wire, it pulls another one off the ring. When there's space on the ring, it pulls another transmit buffer from one of the socket buffers. Or, if there are too many buffers waiting, it drops some of them and then pulls one. So, that part is system-wide (or at least NIC-wide).
So, just knowing your send buffer size doesn't really tell you everything you really care about, and to truly understand it you need to ask questions specific to your kernel. If that's Linux or a *BSD, the networking stack is open source and pretty well documented (in Linux, if I remember correctly, and if my 2.4-era knowledge is still useful, searching for SO_SNDBUF and txqueuelen give you good starting points); otherwise, it may not be. Of course Linux exposes all the information there is to expose somewhere in the /proc filesystem, so once you know what you want to look for, you can find it. *BSD doesn't expose quite as much, but there's a lot of stuff there. Even Windows has a slew of performance counters that are relevant.