Multithreaded access to files in Java

Question

I'm working on a multithreaded server in Java. The server monitors a directory of files. Clients can ask the server:

• to download a file from the server directory
• to upload a new version of an already existing file to the server, overwriting the old version in the server directory.

To do the transfers, I'm planning to use FileChannels and SocketChannels, using the methods transferFrom and transferTo. According to the documentation, these two methods are thread safe. The thing is that a single call to these two function could not be sufficient to read/write the file entirely.

The problem arises if there are more than one request on the same file at the same time. In this scenario, multiple threads could be doing read/write operations on the same file. Now, the single calls to transferFrom/transferTo are thread safe, according to the Java documentation. But a single call to these two functions could not be sufficient to read/write the file entirely. If thread A is replying to a download request and thread B is replying to an upload request referring to the same file, it could happen that:

1. Thread A starts reading from the file
2. In thread A, for some reason the read call returns before the EOF
3. Thread B overwrites the entire file with a single write call
4. Thread A continues reading from the file

In this case, the downloading client receives a portion of the old version and a portion of the new version.

---

To solve this I think I should be using some sort of locking, but I'm not sure how to do it in an efficient way. I could create two synchronized methods for reading and writing, but that creates obviously too much contention.

The best solution I have in mind is to use lock striping. Before doing any read/write operation, an hash based on the filename is calculated. Then, the lock in position lockArr[hash % numOfLocks] is acquired. I think also that I should be using ReadWriteLocks, since multiple simultaneous reads should be allowed.

Now, this is my analysis of the problem and I could be completely wrong. Is there any better solution to this?

Answer 1

Locking means that somebody has to wait for somebody else -- not the best solution.

When the client uploads a file, you should write it out to a temp file on the same disk (usually in the same directory), and then when the file upload is done:

1. Rename the old version to a temporary name. Any current readers should be forced to close the old one, re-open the temp version, and seek to the correct position.
2. Rename the uploaded file to the target file name.
3. Delete the temp version of the old file when any readers are done with it.

In a typical implementation, you'd need a centralized class (lets call it ConcurrentFileAccessor) to manage the interactions between threads.

Readers would need to register with this class, and synchronize on some object during the actual read operation. When an upload completes, the writer would have to claim all those locks to block reads, close all the read files, rename the old version, reopen, seek, and then release them to allow the readers to continue.