This may be slightly OT, but I was wondering why having a process which heavily uses IO (say cp big file from one location to the other on the same disk) slows everything down, even processes which are mostly CPU bound. I noticed that on both OS I heavily use (mac os x and linux).
In particular, I wonder why multi-core does not really help here: is it a hardware limitation for commodity hardware (disk controller, etc...), an os limitation, or is there something inherently hard into allocating the right resources (scheduling) ?
Because, copying a large file (bigger than the available buffer cache) usually involves bringing it through the buffer cache, which generally causes less recently-used pages to be thrown out, which must then be brought back in.
Other processes which are doing tiny small amounts of occasional IO (say just stat'ing a directory) then get their caches all blown away and must do physical reads to bring those pages back in.
Hopefully this can get fixed by a copy-command which can detect this kind of thing and advise the kernel accordingly (e.g. with posix_fadvise) so that a large one-off bulk transfer of a file which does not need to be subsequently read does not completely discard all clean pages from the buffer cache, which now normally mostly happens.