Search code examples
memoryboostmessage-queueinterprocess

Shared memory created by boost message queue is too small?

I am creating an application where several processes are communicating through boost message queues. The queues are created using the message queue constructor as message_queue(open_or_create, name, max_num_msg, max_msg_size); I am using open_or_create for all the queues in all processes since there is not specified order in which the queues must be created. max_num_msg = 200 and max_msg_size = 500000.

Now creation seems all fine, but after sending messages over the queues for a while, I experience sudden crashes due to read access violations. Doing a bit of debugging has taken me to the definition of do_send in message_queue.hpp where an empty message header is obtained for writing the message

  //Insert the first free message in the priority queue
  ipcdetail::msg_hdr_t<VoidPointer> &free_msg_hdr = p_hdr->queue_free_msg(priority);

  //Sanity check, free msgs are always cleaned when received
  BOOST_ASSERT(free_msg_hdr.priority == 0);
  BOOST_ASSERT(free_msg_hdr.len == 0);

The error happens at free_msg_hdr.priority == 0 because the address of free_msg_hdr does not point to a readable location.

Doing a bit more research revealed that in the creation of the queue, 

template<class VoidPointer>
inline message_queue_t<VoidPointer>::message_queue_t(open_or_create_t,
    const char *name,
    size_type max_num_msg,
    size_type max_msg_size,
    const permissions &perm)
    //Create shared memory and execute functor atomically
    : m_shmem(open_or_create,
        name,
        get_mem_size(max_msg_size, max_num_msg),
        read_write,
        static_cast<void*>(0),
        //Prepare initialization functor
        ipcdetail::msg_queue_initialization_func_t<VoidPointer>(max_num_msg, max_msg_size),
        perm)
{}

the created shared memory object m_shmem has a size that is too small to hold 200 messages of size 500000. This explains why the crashes are a bit unpredictable because still a smaller part of the memory is accessible so it takes some time to accidentally get into the inaccessible parts. However, I still have no clue why this is happening. Looking at the function get_mem_size(max_msg_size, max_num_msg) it returns the right size but then after creation the size is smaller. If I then re-create the same queue it usually gets the right size and I never get any exceptions whatsoever. If anyone has any idea of why this could be happening, or suggestions on how to further debug this problem, it would be much appreciated.

I should probably mention that the application is compiled in Visual C++ in 32 bits and runs on Windows 10. Could the Windows shared memory implementation be causing a problem like this?

Solution

  • When the boost library creates a message queue, it requests a piece of shared memory to hold the queue. I don't know exactly how it works and maybe it depends on the OS implementation but on Windows I could see in the debugger that the size of the memory segment is rising slowly (very slowly with the debugger attached, which is why I noticed) while the obtained shared memory is filled with zeroes. In our case, other processes attached to the message queue during this time, apparently obtaining the current size of the shared memory. However, the queue header at the start of the memory tells that it should be larger and therefore the second process will eventually access messages in a memory range that it has not mapped into its address space, generating the read access violation.

    In our case it was relatively easy to make sure that the other processes only open the queue after we are sure that it is created and hence our problem is solved!