Swift DispatchQueue: Serial or parallel

Question

In my app, I have to decompress multiple files in the background, all in the same time. Which code yields a parallel execution of the compressedFiles array on multiple threads:

for file in compressedFiles {
  DispatchQueue.global(qos: .userInteractive).async {
    let work = DispatchGroup()
    work.enter()
    file.decompress()
    work.leave()
  }
}

or:

DispatchQueue.global(qos: .userInteractive).async {
  for file in compressedFiles {
    let work = DispatchGroup()
    work.enter()
    file.decompress()
    work.leave()
  }
}

In addition, how to take advantage of the DispatchGroup class if I want to be notified once one of the files decompression process is finished? Where to put wait() and notify()?

Thanks.

Answer 1

Your second example will run them sequentially. It is doing a single dispatch, running them one after another. Your first example will run them in parallel, dispatching each to a different worker thread. Unfortunately, though, neither is using the dispatch group correctly.

Regarding the dispatch group, you should define it before your loop and enter before you call async. But the manual calling of enter and leave is only needed if you're calling an asynchronous process from within the async call. But given that decompress is likely a synchronous process, you can just supply the group to async, and it will take care of everything for you:

let group = DispatchGroup()

for file in compressedFiles {
    DispatchQueue.global(qos: .userInteractive).async(group: group) {
        file.decompress()
    }
}

group.notify(queue: .main) {
    // all done
}

---

But rather than worrying about the dispatch group logic, there is a deeper problem in the parallel example. Specifically, it suffers from thread explosion, where it can exceed the number of available cores on your CPU. Worse, if you had a lot of files to decompress, you can even exceed the limited number of worker threads that GCD has in its pool. And when that happens, it can prevent anything else from running on GCD for that QoS. Instead, you want to run it in parallel, but you want to constrain it to a reasonable degree of concurrency, while still enjoying parallelism, to avoid exhausting resources for other tasks.

If you want it to run in parallel, but also avoid thread explosion, one would often reach for concurrentPerform. That offers maximum parallelism supported by the CPU, but preventing problems that can result from thread explosion:

DispatchQueue.global(qos: .userInitiated).async {
    DispatchQueue.concurrentPerform(iterations: compressedFiles.count) { index in
        compressedFiles[index].decompress()
    }
    
    DispatchQueue.main.async {
        // all done
    }
}

That will constrain the parallelism to the maximum permitted by the cores on your device. It also eliminates the need for the dispatch group.

---

Alternatively, if you want to enjoy parallelism, but with a lower degree of concurrency (e.g. to leave some cores available for other tasks, to minimize peak memory usage, etc.), you might use use operation queues and maxConcurrentOperationCount:

let queue = OperationQueue()
queue.maxConcurrentOperationCount = 4  // a max of 4 decompress tasks at a time

let completion = BlockOperation {
    // all done
}

for file in compressedFiles {
    let operation = BlockOperation {
        file.decompress()
    }
    completion.addDependency(operation)
    queue.addOperation(operation)
}

OperationQueue.main.addOperation(completion)

---

Or at matt points out, in iOS 13 (or macOS 10.15) and later, you can do:

let queue = OperationQueue()
queue.maxConcurrentOperationCount = 4

for file in compressedFiles {
    queue.addOperation {
        file.decompress()
    }
}

queue.addBarrierBlock {
    DispatchQueue.main.async {
        // all done
    }
}