Why a mutex can be locked by two threads?

Question

My C program has three threads A, B, and C that share some buffers. Each buffer is protected by a mutex, and each thread has to lock the mutex before writing/reading a buffer.

A writes the buffers, and B and C reads the buffers. The main function creates A, waits until it locks the first buffer with pthread_mutex_lock(), and then creates B and C, so that the buffers are always written before being read.

• main.c

#define mutex_lock(mutex, buff_num)        log("wait on buff[%d]\n", buff_num);  \
                                           printf("pthread_mutex_lock returns %d\n", pthread_mutex_lock(mutex));   \
                                           log("buff[%d] locked\n", buff_num)
    
#define mutex_unlock(mutex, buff_num)      log("releasing buff[%d]\n", buff_num); \
                                           pthread_mutex_unlock(mutex);           \
                                           log("buff[%d] released\n", buff_num)

...
typedef struct
{
    pthread_mutex_t  lock;
    char             data[MAX_DATA_LENGTH];
} buff_t;

uint8_t A_thread_ready = 0;
buff_t buff[NUM_BUFF];
...

int main()
{
    ...
    for(int i=0; i<NUM_BUFF; i++)
    {
        if (pthread_mutex_init(&buff[i].lock, NULL) != 0)
        {
            pthread_exit(NULL);
        }
    }
    ...
    status = pthread_create(&tid[0], NULL, &A_thread, NULL);
    if ( 0 == status)
    {
        break;
    }

    do
    {
        nanosleep(&t1, &t2);
    } while (0 == A_thread_ready);

    status = pthread_create(&tid[1], NULL, &B_thread, NULL);
    if ( 0 == status)
    {
        break;
    }
    ...
}

• a.c

...

extern uint8_t A_thread_ready;
extern buff_t buff[NUM_BUFF];

...

void* A_thread(void* arg)
{
    uint8_t write_buff = 0;
    ...
    mutex_lock(&buff_p->lock, write_buff);
    buff_p = &(buff[write_buff]);
    A_thread_ready = 1;
    while(1)
    {
        ...
        mutex_unlock(&buff_p->lock, write_buff);
        write_buff++;
        write_buff %= NUM_BUFF;
        mutex_lock(&buff_p->lock, write_buff);
        buff_p = &(buff[write_buff]);
        ...
    }
    ...
}

• b.c

...

extern buff_t buff[NUM_BUFF];

...

void* B_thread(void* arg)
{
    uint8_t read_buff = 0;
    ...
    mutex_lock(&buff_p->lock, read_buff);
    buff_p = &(buff[read_buff]);

    while(1)
    {
        ...
        mutex_unlock(&buff_p->lock, read_buff);
        read_buff++;
        read_buff %= NUM_BUFF;
        mutex_lock(&buff_p->lock, read_buff);
        buff_p = &(buff[read_buff]);
        ...
    }
    ...
}

When running the program, however, from the printed logs, it seems that after A locked a buffer, B was able to lock this same buffer.

It did something, and released the buffer. After some time A released this buffer. I checked the return value of each pthread_mutex_lock() call and it was always 0 (successful). What else should I check to find the cause of this problem?

Answer 1

At first glance I'm deeply concerned about this code:

    mutex_lock(&buff_p->lock, read_buff);
    buff_p = &(buff[read_buff]);

    while(1)
    {
        ...
        mutex_unlock(&buff_p->lock, read_buff);

It sure looks like you lock some mutex stored in a structure at buff_p->lock (buffer 1's lock), then you change buff_p's value with buff_p = &(buff[read_buff]);, and then you unlock some other mutex: the mutex at buff_p->lock (buffer 2's lock). Presumably those are different locks on completely different buffers.