Question about condition_variable, why condition_variable is paired with mutex

I have been learning std::condition_variable recently and there is just some question that I cannot understand.

Cppreference and many other tutorials give example like this:

std::mutex m;
std::condition_variable cv;

void worker_thread()
{
    // wait until main() sends data
    std::unique_lock lk(m);
    cv.wait(lk, []{ return ready; });
 
    // after the wait, we own the lock
    std::cout << "Worker thread is processing data\n";
    data += " after processing";
 
    // send data back to main()
    processed = true;
    std::cout << "Worker thread signals data processing completed\n";
 
    // manual unlocking is done before notifying, to avoid waking up
    // the waiting thread only to block again (see notify_one for details)
    lk.unlock();
    cv.notify_one();
}

Why is condition_variable always paired with mutex? I can't understand the reason why mutex is needed here.
What is the lambda function []{ return ready; } doing here?

Solution

std::condition_variable exists as a tool to do some low level messaging. Instead of providing a "semaphore" or a "gate" or other threading primitives, the C++ std library provided a low level primitive that matches how hardware threading works and you can use to implement those other threading primitives.

std::condition_variable provides the hooks to have notification occur, while std::mutex provides the hooks to guard data. As it happens, in the real world of actual hardware and OS provided notification primitives, the notification primitives hardware provide are not 100% reliable, so you need to have some data to back up your notification system.

Specifically, spurious notifications are possible - a notification can occur that doesn't correspond to anyone sending a notification to your std::condition_variable (or its underlying hardware/OS primitive).

So when you get a notification, you must check some data (in a thread safe way) to determine if the notification actually corresponded to a message or not.

The result is that the standard way to use std::condition_variable is to have 3 tightly related pieces:

The std::mutex which guards some data that contains a possible message.
The std::condition_variable which is used to transmit the notification.
The data that contains the message itself.

A really simple message system might be a gate that can be opened and never closed. Here, your data is a bool that states "is the gate open". When you open the gate, you modify that bool (in a thread-safe manner) and notify anyone waiting for the gate to open. Code waiting for the gate to open waits on the condition variable; when it wakes up, it checks if the gate it open, and only accepts the notification if it is actually open. If the gate is closed, it considers the wake up spurious, and goes back to sleep.

In actual code:

struct Gate {

  void open() {
    auto l = lock();
    is_open = true;
    cv.notify_all();
  }

  void wait() const {
    auto l = lock();
    cv.wait(l, [&]{return is_open;});
  }

private:
  mutable std::mutex m;
  bool is_open = false;
  mutable std::condition_variable cv;

  auto lock() const { return std::unique_lock{m}; }
}

in open, we lock the mutex (because we are editing shared state - the bool), we edit the bool is_open to say it is open, then we notify everyone who is waiting on it being open that it is indeed open.

On the wait side, we need a mutex to call cv.wait. We then wait until the gate is open -- the lambda version of wait builds a loop for us that checks for spurious wakeups and goes back to sleep when they happen.

The lambda version:

 auto l = lock();
 cv.wait(l, [&]{return is_open;});

is just short hand for:

 auto l = lock();
 while (!is_open)
   cv.wait(l);

ie, a little "wait loop" looking for is_open to be true.

Without cv.wait(l) the code would be a busy-loop (well, you'd also want to unlock l and relock it). With cv.wait(l), it will unlock the mutex m and wait for a notify to occur; the thread won't spin. When a notify happens (or sometimes spuriously -- for no reason whatsoever) it will wake up, reget the lock l on mutex m, then check is_open. If it is actually open, it will exit the function; if it isn't open, it will chalk it up to being a spurious notification, and loop.

You can write a whole bunch of primitives for notifications using this condition variable primitive - thread safe message queues, gates, semaphores, multi-future waiting systems, etc.

My favorite tool to do this looks like this:

template<class T>
struct mutex_guarded {
  auto read(auto f) const -> decltype( f(std::declval<T const&>()) ) {
    auto l = lock();
    return f(t);
  }
  auto write(auto f) -> decltype( f(std::declval<T&>()) ) {
    auto l = lock();
    return f(t);
  }
protected:
  mutable std::mutex m;
  T t;
};

this is a little pseudo-monad that wraps an arbitrary object of type T in a mutex. We then extend it:

enum class notify {
  none,
  one,
  all
};

template<class T>
struct notifier:mutex_guarded<T> {
  notify maybe_notify(auto f) { // f(T&)->notify
    auto l = this->lock();
    switch( f(this->t) ) {
      case notify::none: return notify::none;
      case notify::one: cv.notify_one(); return notify::one;
      case notify::all: cv.notify_all(); return notify::all;
    }
  }
  void wait(auto f) const { // f(T const&)->bool
    auto l = this->lock();
    cv.wait(l, [&]{ return f(this->t); });
  }
  bool wait_for(auto f, auto duration) const; // f(T const&)->bool
  bool wait_until(auto f, auto time_point) const; // f(T const&)->bool
private:
  mutable std::condition_variable cv;
};

here we wrap up the notification code as in a pseudo-monad.

notifier<bool> gate;
// the open() method is:
gate.maybe_notify([](bool& open){open=true; return notify::all;});
// the wait() method is:
gate.wait([](bool open){return open;});

This covers about 99% of uses of std::condition_variable and std::mutex; the remaining 1% is ... more advanced.