Using "withTaskGroup" and "async let" together

Question

Following is a simplified version of how I'm using withTaskGroup:

let allResults = await withTaskGroup(of: ([Double], Float, Float, Int).self,
                                     returning: FinalResult.self,
                                     body: { taskGroup in

    for operation in operations {
        taskGroup.addTask { @MainActor in
            let value = await operation.execute()
            return value
        }
    }
    
    // Collect child task results here...
})

The execute function looks as follows where the processes include fairly math calculation intensive tasks such as feature extractions for a machine learning model:

func execute() async -> ([Double], Float, Float, Int) {
    async let result0: [Double] = process0() ?? []
    async let result1: Float = process1() ?? -1
    async let result2: Float = process2()
    async let result3: Int = process3()
    return (await result0, await result1, await result2, await result3)
}

The issue I'm experiencing is that the final result is inconsistent where sometimes I get proper results and sometimes not. An example would be that sometimes I get an array of all zeros for 512 features (array length) when they're supposed to be non zero numbers. Contrarily, if I were to simply use a for loop instead of withTaskGroup, the result seems to be a bit more consistent.

So I'm wondering if there are any consequences of using withTaskGroup and async let together since this is a parallel of a parallel. Unlike concurrency, my understanding of parallelism is that each core of CPU is assigned to process the tasks simultaneously, but if I were to provide 4 operations, that means 4 x 4 = 16 parallel processes, which is more than the number of available cores.

Answer 1

This is definitely not the problem. The problem is much more likely that your code in process1(), etc., is not thread-safe (relying on data read from properties that aren't thread-safe, for example, or manipulating the returned Array in an invalid way). Exceeding the number of cores is not going to cause a logic problem if you've otherwise written the code correctly. It can cause performance problems, depending on many factors, but it will not cause problems with correctness.

You may want to turn on "Strict Concurrency Checking" in your build settings, which can help you find mistakes. (It can also find a lot of issues in Foundation which you can't fix, so it's sometimes best to use it on isolated code or a smaller project.)