Search code examples
c++multithreadingtimeoutthread-safety

Best approach to Independently Timing each Tick of a For Loop in another Thread


Suppose I have a client thread and a server thread. The client thread must perform an expensive for loop operation which is prone to hanging. Thus, the server has independently determine whether each tick of the for loop has exceeded the max time. The context behind this is that the server will timeout the client if it takes too long to complete a tick.

My initial idea below is to have two for loops in the client and server thread. The server thread will have a condition variable that waits for 1 second. If the client does not notify the condition variable in 1 second every tick, the server will time it out:

Server

 bool success;
 for (int i = 0; i < 10; i++) {
     std::unique_lock<std::mutex> lock(CLIENT_MUTEX);
     success = CLIENT_CV.wait_for(lock, std::chrono::seconds(1));
     if (!success) {
         std::cout << "timed out during tick " << i << std::endl;
         break;
     }
  }

Client

for (int i = 0; i < 10; i++) {
    std::unique_lock<std::mutex> lock(CLIENT_MUTEX);

    //do work

    CLIENT_CV.notify_one();
}

However my implementation attempt is unreliable and times out at random times given the same work for the client. How can I improve the design to make it more reliable?

Side Note:

A simple solution to this would be for the server to time the entire for loop as opposed to each tick. However if the for loop fails on tick 1 out of 10, and the timer is waiting for 10 seconds, then the client will be informed after 10 seconds. However if the server was to impose a 1 second timeout for each tick (10x1sec = 10secs) then the client will be informed of timeout without having to wait the full 10 seconds.

Edit.

This whole client/server/timeout analogy is simply to put the question into context. I'm purely interested in the best way to time the for loop from a different thread.


Solution

  • One way of doing this might be:

    Shared vars:

    std::vector<std::chrono::time_point<std::chrono::high_resolution_clock>> ledger;
    std::mutex ledger_mtx;
    

    Client:

    for (int i = 0; i < 10; i++) {
        {
        std::scoped_lock lock(ledger_mtx);
        ledger.push_back(std::chrono::high_resolution_clock::now());
        }
        // Do work
    }
    {
    std::scoped_lock lock(ledger_mtx);
    ledger.push_back(std::chrono::high_resolution_clock::now());
    }
    

    Server:

    size_t id = 0;
    std::this_thread::wait_for(1s); // Some time so that initial write to ledger is made
    while(true) {
        {
        std::scoped_lock lock(ledger_mtx);
        if(ledger.size()==id) { /* Do something if the thread hangs */ }
        id = ledger.size();
        std::chrono::time_point<std::chrono::high_resolution_clock> last_tick = ledger.back();
        }
        if(id == 11) break;
        std::this_thread::sleep_for(1s - (std::chrono::high_resolution_clock::now() - last_tick));
    }
    

    This way you can time the thread, while monitoring it from the outside. Is it the best way? probably not, but it does give you the times you need.