How to guarantee that the API returns before its callback is invoked in asyncronous API (c++)

Question

I develop a library which has asyncronous APIs. One of the APIs, when it is called, it creates a task, pushs it to task thread and returns its task ID. After the task is completed, task thread notify the result to caller by invoking a callback function.

The sequence is as follows

Caller

C.1. A Caller Calls API

C.2. Lib. create task and push it to the queue of the Task Thread

C.3. Lib. awakes the Task Thread by calling notify_all of a condition_variable

C.4. At this point, context switch can occur and this thread will be suspended

C.6. After this thread is resumed, Lib. returns task ID

Task Thread

T.1. The Task Thread executes the task.

T.2. When the task is completed, The Task Thread notifies the result to caller by invoking the callback

Problem

A caller check the result data of the callback function by task ID, but ocasionally the callback is invoked before the API returns and caller cannot check the result.

Question

I want to guarantee perfectly that the API returns task ID before its callback is invoked. What can I do?

I use lock_guard in the API's body to prevent that the callback is invoked, and it diminishes the possibility to reproduce this problem significantly.

But because lock_guard unlocks the mutext before API returns, if context-switch occurs after mutex unlocks, before API returns, this problem can be reproduced very rarely.

I want to prevent this case too.

summarized codes

long AcbCore::setState(long appState, long playState)   // API
{
    StateTask* task = (StateTask*) createTask(TaskType::PLAYER_STATE_CHANGE, &isAppSwitchingStateFlag, appState);

    std::lock_guard<std::mutex> lockGd (*task->getEventMutex());
    pushTask(task);      // at this position, context switch can occur by condTaskThread_.notify_all() of resumeLoop()

    return taskId;
}

void AcbCore::pushTask(Task* task)
{
    mtxTaskQueue_.lock();
    queueTask_.push_back(task);
    mtxTaskQueue_.unlock();

    resumeLoop();
}

void AcbCore::resumeLoop()
{
    mtxTaskThread_.lock();
    mtxTaskThread_.unlock();
    condTaskThread_.notify_all();
}

bool AcbCore::suspendLoop()
{
    bool isTimeout = false;
    if (ingTask_ != NULL) {
        isTimeout = (condTaskThread_.wait_for(lockTaskThread_, std::chrono::seconds(AWAKE_TASK_THREAD_TIMEOUT)) == std::cv_status::timeout);
    } else {
        condTaskThread_.wait(lockTaskThread_);
    }

    return isTimeout;
}

void AcbCore::taskLoop()  // loop of Task Thread
{
    Task* task = NULL;
    Action* action = NULL;
    while (isInitialized_) {
        while (popTask(task)) {
            if (task->isCompleted()) {
                fireEvent(task);
            } else {
                doNextTask(task);
            }
        }
        if (suspendLoop()) {    //  if awaked by timeout
            cancelTask(ingTask_, true);
        }
    }
}

void AcbCore::fireEvent(Task* task, bool bDelete)
{
    std::string errorInfo = task->getErrorInfo();

    task->waitToUnlockEvent();
    // eventHandler_ : callback set by caller when Acb is initialized
    eventHandler_(task->getTaskId(), task->getEventType(), appState_.now_, playState_.now_, errorInfo.c_str());

    if (bDelete) {
        deleteTask(task);
    }
}

Answer 1

Fundamentally, you can't solve this on your side. Let's assume that the initial thread is suspended exactly after the last instruction of your code, but before the caller gets the task ID. You simply cannot determine when the caller has stored the task ID in a way that allows the callback to happen.

Instead, the client should cache unknown task IDs when he's got an outstanding call to your async function.