we are receiving about 10000 messages per hour. We store them as individual files in hourly directories on an ext3 filesystem. The file name includes a sequence number. We use rsync to mirror these files every 20 seconds at another location (via a SAN, but that doesn't matter).
Sometimes an rsync run picks up files n-3, n-2, n-1, n+1, and then next rsync run continues with n, n+2, n+3, n+4 and so on.
Is it possible that when one process creates files in a certain sequence within a directory, that another process using readdir() sees the files appearing in a different sequence?
Kind regards, Sebastian
I suppose your question can be restated as:
If process A creates file
d/xand then creates filed/y, is it possible for process B to peform a concurrentreaddir()on directorydand see an entryd/y, but not see an entryd/x?
The answer is Yes. The ordering guarantees for readdir are very weak indeed.
If you want to enforce an ordering, you will need to explicitly fsync() a file descriptor for the directory d itself after creating each file.