I have never tried this but it's something I was wondering about. If I am downloading a very big file (say 200 GB) using very fast links (1 Gbps or even 10 Gbps), how does the SO (or whoever do this) writes the downloaded file at the same time in the disk, since disks have very slow write speed compared to my link speed? Would in this case, the hard drive become a bottleneck? If I run iostat in my PC it shows 1027 KBps ~ 1 MBps write speed, which is very slow compared with the link stated before.
Yes, it's certainly possible for almost any part of your hardware chain, from the incoming link to your hard drive to become the bottleneck, depending on your hardware.
In the case that you are actually sustaining a download speed faster than the linear write speed of your hard drive, it could certainly become a bottleneck.
Note, however, that even most budget hard drives today1 have a linear write speed of at least 50 MB/s, which is 400 Mbps, and often closer to 100 MB/s (i.e., 800 Mbps). At an effective write speed of 800 Mbps, your drive should be able to keep up with a saturated 1 Gbps link2, at least approximately, but would certainly fall behind against a 10 Gbps link.
Now, what you were measuring with iostat
isn't any kind of useful benchmark - it's telling you the actual throughput for either the entire uptime of your host of active requests over some interval: unless are doing a large transfer during the interval, the speeds reported there have little relationship to your actual disk write speeds. There are plenty of benchmarking tools that will measure directly your read and write speeds.
A final think to keep in mind is that most modern operating systems use a "write-back" strategy for storage writes - the writes are first sent to RAM (i.e., the file cache on Windows or page cache on Linux), and then streamed out to the disk asynchronously. This helps hide the throughput of the actual disk for relatively short bursts of writes. For example, your disk may appear to have a very high throughput of 5 GB/s or more if writes are of a few hundred MBs or a few GBs3, but then will approach the true disk speed as they get larger, since the buffering ability of your OS disk cache will be exhausted. Clearly for a 200 GB transfer, the disk cache isn't going to be able to hide the disk speed, unless you have 100s of GBs of RAM.
So all that said, yes, your harddrive can certainly become a bottleneck,
but likely at high higher throughputs than what you measured with iostat
.
1 This increase in linear read and write speeds is mostly an artifact of increasing areal density of storage, which directly translates to increased linear read/write speeds at the same RPM. Random read/write has no such relationship, however. SSDs of reasonable capacity usually increase this by another order of magnitude, to around 1 GB/s (8 Gbps) or more.
2 Mostly because these network links have overhead which for fast links often reduces the actual payload to less than 80% of the theoretical link speed.
3 The exact values depend on your total RAM, available RAM and OS configuration.