Why does DispatchSemaphore.wait() block this completion handler?

Question

So I've been playing about with NetworkExtension to to make a toy VPN implementation and I ran into an issue with the completion handlers/asynchronously running code. I'll run you through my train of thought/expirments and would appreciate any pointers at areas where I am mistaken, and how to resolve this issue!

Here's the smallest reproducible bit of code (obviously you will need to import NetworkExtension):

let semaphore = DispatchSemaphore(value: 0)
NETunnelProviderManager.loadAllFromPreferences { managers, error in
    print("2 during")
    semaphore.signal()
}
print("1 before")
semaphore.wait()
print("3 after")

With my understanding of semaphores and asynchronous code I'd expect the printouts to occur in the order:

1 before
2 during
3 after

However the program hangs at "1 before". If I remove the semaphore.wait() line, the printout occurs as expected in the order: 1, 3, 2 (as the closure runs later).

So after a bit of digging around with the debugger, it looks like the semaphore trap loop is blocking up execution. This sparked me to read around a bit into queues, and I discovered that changing it to the following works:

// ... as before
DispatchQueue.global().async {
    semaphore.wait()
    print("3 after")
}

This makes some sense as the blocking .wait() call is now being called asynchronously in a separate thread. However, this solution is not desired for me as in my actual implementation I am actually capturing the results from the closure and returning them later, in something that looks like this:

let semaphore = DispatchSemaphore(value: 0)
var results: [NETunnelProviderManager]? = nil
NETunnelProviderManager.loadAllFromPreferences { managers, error in
    print("2 during")
    results = managers
    semaphore.signal()
}
print("1 before")
// DispatchQueue.global().async {
    semaphore.wait()
    print("3 after")
// }
return results

Obviously I cannot return data from from the async closure, and moving the return out of it would make it defunct. Acdditionally, adding another semaphore to make things synchronous exhibits the same issue as before just moving the problem along in a chain.

As a result, I decided to try putting the .loadAllFromPreferences() call and completion handler in an async closure and leave everything else as in the original code snippet:

// ...
DispatchQueue.global().async {
    NETunnelProviderManager.loadAllFromPreferences { loadedManagers, error in
        print("2 during")
        semaphore.signal()
    }
}
// ...

However this does not work and the .wait() call is never passed - as before. I assume that somehow the sempahore is still blocking the thread and not allowing anything to execute, meaning whatever in the system is managing the queue is not running the async block? However I'm clutching at straws here, and fear my original conclusion may not have been right.

This is where I'm starting to get out of my depth, so I'd like to know what is actually going on, and what resolution would you recommend to get the results from .loadAllFromPreferences() in a synchronous manner?

Thanks!

Answer 1

From the documentation for NETunnelProviderManager loadAllFromPreferences:

This block will be executed on the caller’s main thread after the load operation is complete

So we know that the completion handler is on the main thread.

We also know that the call to DispatchSemaphore wait will block whatever thread it is running on. Given this evidence, you must be calling all of this code from the main thread. Since your call to wait is blocking the main thread, the completion handler can never be called because the main thread is blocked.

This is made clear by your attempt to call wait on some global background queue. That allows the completion block to be called because your use of wait is no longer blocking the main thread.

And your attempt to call loadAllFromPreferences from a global background queue doesn't change anything because its completion block is still called on the main thread and your call to wait is still on the main thread.

It's a bad idea to block the main thread at all. The proper solution is to refactor whatever method this code is in to use its own completion handler instead of trying to use a normal return value.