Search code examples
c++multithreadingpthreadsposixproducer-consumer

pthread_mutex_wait multiple producer and consumer


I was going through this and this link. Basically, they are discussing why classical single threaded producer consumer design (with signal and wait does not work for multi-producer scenario). I have one doubt which has been bugging me -

Argument made by author
Consider reference code

char queue[MAX];  //global
int head = 0, tail = 0; //global
struct cv not_full, not_empty;
struct lock qlock;

void produce(char data) {
  acquire(&qlock);
  if ((head + 1) % MAX  ==  tail) {
    wait(&not_full, &qlock);   //line 1
  }
  queue[head] = data; //line 2
  head = (head + 1) % MAX;
  notify(&not_full);
  release(&qlock);
}

char consume(void) {
  acquire(&qlock);
  if (tail == head) {
    wait(&not_empty, &qlock);
  }
  e = queue[tail];
  tail = (tail + 1) % MAX;
  notify(&not_empty);
  release(&qlock);
  return e;
}

In the above code, in case of two producer, line 1 will be woken up in both the producer thread by single consumer (when queue is not full). So both producer can add to the queue leading to queue overflow.

My doubt
a. We are protecting the queue with mutex lock. So even if wait is woken up on multiple producer thread, only one producer would still have mutex lock - so logically there would be only one producer 'owning' rights to add to queue. Because when we come out of wait we would have mutex to be acquired.

Cavet
I use POSIX mutex,cond var as reference. However, I do not see article written with POSIX as reference standard.

Question
Is my understanding of wait specifically pthread_cond_wait correct? And with multiple producers, the integrity of the code is still maintained.


Solution

  • The author mentionned at the end of its article under Semantics of wait() and notify() that

    notify(cv) wakes up all threads that are currently waiting on that cv

    So your understanding of wait is incorrect, notify is meant to be pthread_cond_broadcast in posix.

    Furthermore the documentation of pthread_cond_signal stipulates

    The pthread_cond_signal() call unblocks at least one of the threads that are blocked on the specified condition variable cond (if any threads are blocked on cond).

    Which is still different from your "only one producer" assumption.

    As the author shown, the integrity of the code above is not maintained with multiple producers.

    edit:

    The pseudocode of a condition variable wait may look like

    void wait (condition *cv, mutex *mx) 
    {
        mutex_acquire(&c->listLock);  /* protect the queue */
        enqueue (&c->next, &c->prev, thr_self()); /* enqueue */
        mutex_release (&c->listLock); /* we're done with the list */
    
        /* The suspend and release_mutex() operation should be atomic */
        release_mutex (mx));
        thr_suspend (self);  /* Sleep 'til someone wakes us */
    
        <-------- notify executes somewhere else
    
        mutex_acquire (mx); /* Woke up -- our turn, get resource lock */
        return;
    }
    

    during a signal at least one threads in the queue in suspend state is awaken. But pthread doesnt guarantee it will be only one. Now they are runnable. But they still need to acquire the lock to ensure mutual exclusion between each other.

    So when the first releases the mutex, the second takes it and so on.

    Which means that one after the other the awaken producers will execute

    queue[head] = data; //line 2
    head = (head + 1) % MAX;
    notify(&not_full);
    release(&qlock);