Publisher not calling completion when test is async (case for M1 processor)

Question

I have a class that can load data asynchronously from the file system. I am trying to test reading the data correctly and have been seeing how the publisher I create to call the loading task only finishes as long as the test is not marked as async.

I need the test to be async to be able to await saving the data to the file system before testing reading it. For the sake of demonstrating the failure, I have omitted that part. These are the tests I am running, one passing, the other one failing:

func testPassing_localSavedLocallyIsUsedWhenAvailable() {
    let url = testSpecificStoreURL()
    let (publisher, _) = makeSUT(url: url)

    let expectedData = waitForPublication(on: publisher) ?? Data()
    XCTAssertEqual("Example data", String(data: expectedData, encoding: .utf8))
}

func testFailing_localSavedLocallyIsUsedWhenAvailable() async {
    let url = testSpecificStoreURL()
    let (publisher, _) = makeSUT(url: url)

    let expectedData = waitForPublication(on: publisher) ?? Data()
    XCTAssertEqual("Example data", String(data: expectedData, encoding: .utf8))
}

Notice how they are identical, except for the func testFailing... one being async.

The failing one fails with message:

.../TestingCombine/TestingCombineTests/CombineHelperTests.swift:93: error: -[TestingCombineTests.CombineHelperTests testFailing_localSavedLocallyIsUsedWhenAvailable] : Asynchronous wait failed: Exceeded timeout of 3 seconds, with unfulfilled expectations: "Wait for publisher".
.../TestingCombine/TestingCombineTests/CombineHelperTests.swift:70: error: -[TestingCombineTests.CombineHelperTests testFailing_localSavedLocallyIsUsedWhenAvailable] : XCTAssertEqual failed: ("Optional("Example data")") is not equal to ("Optional("")")

The helper methods testSpecificationURL(), makeSUT(url:) and waitForPublication can be found in this repository: https://github.com/lfcj/TestingCombine/blob/main/TestingCombineTests/CombineHelperTests.swift

It is a repo with only one important file to be able to reproduce.

The repo even has GitHub Actions and the async test also fails. The logs are very scarce for GitHub Actions, but the failure happens after timeout + a couple of milliseconds every time, example:

When timeout is 3

Test case 'CombineHelperTests.testFailing_localSavedLocallyIsUsedWhenAvailable()' failed on 'Clone 1 of iPhone 14 - TestingCombine (4572)' (3.114 seconds)

When timeout is 5

Test case 'CombineHelperTests.testFailing_localSavedLocallyIsUsedWhenAvailable()' failed on 'Clone 1 of iPhone 14 - TestingCombine (3784)' (5.194 seconds)

When timeout is 10

Test case 'CombineHelperTests.testFailing_localSavedLocallyIsUsedWhenAvailable()' failed on 'Clone 1 of iPhone 14 - TestingCombine (6248)' (10.121 seconds)

Any idea what is causing this to happen? A teacher of mine cannot reproduce on an Intel machine, I can reproduce each time on my M1.

Update: I was able to reproduce the issue on two different M1 devices every time. Not a single time for an Intel Machine.

Answer 1

The problem is that we are "waiting" for the expectation, not awaiting for it. Using:

@MainActor func waitForExpectations(timeout: TimeInterval, handler: XCWaitCompletionHandler? = nil)

takes care of waiting until the Task is done. More information can be found in Apple Forum.

So the solution is doing:

func testFailing_localSavedLocallyIsUsedWhenAvailable() async {
        let url = testSpecificStoreURL()
        let (publisher, _) = makeSUT(url: url)

        let exp = expectation(description: "Wait for publisher")
        var expectedData = Data()
        publisher.sink(
            receiveCompletion: { completion in
                exp.fulfill()
            },
            receiveValue: { receivedData in
                expectedData = receivedData
            }
        ).store(in: &cancellables)
        
        await waitForExpectations(timeout: 3)
        XCTAssertEqual("Example data", String(data: expectedData, encoding: .utf8))
}