Search code examples
linuxwindowsiodiskdd

Who determine the block size when writting to a disk?


This might be a naive question but I can't find a straight answer for it.

While using IO tools such as dd tool, fio and bonnie++, one of the tools parameters is to set the block size that will be used in the test. So, one can set the block size to 512 KB, 1 MB or even more. And as the block size get greater the output MB/s also get higher, and I believe it is logical, since you get to write on less blocks.

So my questions are:

-How does the process happen while the default block size is 4 KB or 32 KB in some kernels ?!

-In any other application, who determine the block size to write on a disk ? is it the application itself or the operating system ?!

-What would be a typical block size of a database application for instance ?!

Thanks in advance :)


Solution

  • If you use something like dd, you're doing a block-level operation, so you get to specify a block size. Up to a point, you'll get greater speed by using a bigger block size, but it will quickly tail off. It's very inefficient to read from a disk byte by byte, but by the time you've hit a few megabytes, you won't notice any further speed increase.

    When an application writes to disk, it is generally not doing block-level access, but reading and writing files. It's the operating system that is responsible for turning this file-level access into block-level access. An application, unless it's a specialised one running as root, won't care about block-level access, and won't be involved in determining block sizes for this kind of thing.

    It's further complicated by the disk cache: when you read something at the application level, if you're lucky, you won't touch the disk at all: it'll be something already cached, and you'll retrieve it from there (without being aware of it). When you write, you will hopefully find that you write into the cache and appear to finish immediately, and then the operating system will do the actual write when it gets round to it. It's only if you're doing lots of writing, or if the cache is turned off, that you will exhaust the cache and the writes will need to happen before control gets passed back to your application.

    In short: unless you muck about at a fairly low level, you don't need to worry about block sizes.