Search code examples
ioaiofsync

What will be if I use libaio + fsync()?

I know that when I write files using write() + fsync() (or O_SYNC + write(), I think they are same ref #1 ref #2), it means I am using blocking synchronous I/O, and the if the write()(with O_SYNC) or fsync() returns, it means the data is safely on the device medium (e.g., TLC NAND for SSD) rather than the device cache (e.g., DDRAM in SSD).

While what if I use libaio? (because I want to make sure the write issued by libaio is on the storage medium rather than the device cache. i.e., I suppose when io_getevents() returns, it may not make sure the write is on the storage medium, and it may just on the device cache)

  • Question 1: does fsync() exclusively works for synchronous I/O?
  • Question 2：is fsync() after io_submit() an undefined behavior?
  • Question 3: how to make asynchronous write safely persisted to the device medium rather than device cache (no battery-backed cache).
Solution

  • (This answer is only looking at things from Linux perspective. Other operating systems may have different behaviour/semantics)

    So long as you wait for any outstanding asynchronous I/O to be completed you will be able to then send down your fsync() and know all previous I/O was written to stable storage to the best knowledge of the Linux kernel*.

    does fsync() exclusively works for synchronous I/O?

    This question doesn't make sense. fsync() works on I/O that Linux has agreed has been "sent" and then also flushes the device cache. If you asynchronously send an I/O to the kernel, has the kernel agreed it has been "sent"? Also see the answer to final question...

    is fsync() after io_submit() an undefined behavior?

    Technically undefined behaviour would mean from then on ANYTHING would be allowed to legally happen (e.g. so called nasal daemons). That's not the case here but I think you are asking your first question so the answer there still holds.

    how to make asynchronous write safely persisted to the device medium rather than device cache (no battery-backed cache).

    If you're stuck using libaio you could manually build a barrier by waiting for all outstanding I/O to be completed and then sending an fsync() and waiting for that or (as mentioned in a comment). Another libaio technique is to set the RWF_SYNC flag during io_submit(). If you were using io_uring you would have another option because you can use its chaining and it has an asynchronous fsync operation (IORING_OP_FSYNC).

    * I'm excluding the cases where errors happen because they are rather more complicated. See Writing programs to cope with I/O errors causing lost writes on Linux for a detailed explaintion.