Multithread: Dynamically object locking based on resource in hava

Question

Need threads expert eyes here...

I am on a poc app where i am uploading files on FTP server

In FTP server have multiple folders. Based on input response I reads files from the folder and move to another folder

The app can access by multiple threads at a time.

So the problem was this:

Suppose FTP have a folders Folder_A and A_A_FOLDER Now Folder_A have 10 files. a thread came and read 10 files from FTP and start some calculation on it, it calculated one by one and then move to A_A_FOLDER it was middle in the process (let suppose it successfully moved 5 files from Folder_A to A_A_FOLDER) then another thread came and it picks remaining 5 files because they were underprocessed by thread 1, so thread 2 also start processing those 5 files

So duplicate files problem here

void m1(String folderName) {
// FTP related code
}

I have solved this problem by using synchronized keyword

Now every thing in sync and all processing working fine

synchronized void m1(String folderName) {
// code
}

folderName decide which folder need to process

Now I have started facing performance issue

because the method is synchronized so all thread going to wait until processing thread not completed its task.

I can improve this by following steps:

(Before going to a solution here is the some story to much dig on the problem)

As I have mentioned folderName parameter of m1 method decide which folder will process, So suppose I have 4 folders (A, B, A_T, B_T) in Ftp server, 2 folders are those where data need to read from (A and B), And 2 folder are those where data will move (A_T and B_T)

A_T and B_T is not the concern here because they are unique for each folder A and B So if the method will read from A then it will move it to A_T same for B (move to B_T)

Now:

Suppose 4 thread comes to m1 method, 3 threads for folder A and 1 for folder B if somehow method synchronized request based on fileName parameter so I can improve the performance, means 1 thread will work on A another 2 threading will block because fileName is same for them so they will wait until first thread not completed it task where thread 4 will parallel work without any locking process because it's file name is different

So how can I achieve this(synchronized on fileName) in code level?

Note: i know i can break this logic using static locking list for resource and then the locks fileName resource e.g:

private final Object A = new Object();
private final Object B = new Object();

but the problem with this approach is folder can be dynamically added, so I can't go with this.

Need your help guys.

Answer 1

One approach would to be maintain a lock per directory:

public class DirectoryTaskManager {
    public static void main(String[] args) throws IOException {
        DirectoryTaskManager manager = new DirectoryTaskManager();
        manager.withDirLock(new File("Folder_A"), () -> System.out.println("Doing something..."));
    }

    public void withDirLock(File dir, Runnable task) throws IOException {
        ReentrantLock lock = getDirLock(dir);
        lock.lock();
        try {
            task.run();
        } finally {
            lock.unlock();
        }
    }

    private Map<File, ReentrantLock> dirLocks = Collections.synchronizedMap(new HashMap<>());

    public ReentrantLock getDirLock(File dir) throws IOException {
        // Resolve the canonical file here so that different paths 
        // to the same file use the same lock
        File canonicalDir = dir.getCanonicalFile();
        if (!canonicalDir.exists() || !canonicalDir.isDirectory()) {
            throw new FileNotFoundException(canonicalDir.getName());
        }
        return dirLocks.computeIfAbsent(canonicalDir, d -> new ReentrantLock());
    }
}