Iterators, value of result of call to end() changes as data is added to circular_buffer

Question

I discovered earlier today that the iterators of a boost::circular buffer were not behaving quite as I expected them to in a multi-threaded environment. (although to be fair they behave differently than I would think in a single threaded program too).

if you make calls to buffer.begin() and buffer.end() to represent iterators to use to loop over a specific seqment of data. The value of the end iterator changes if more data is added to the circular_buffer. Obviously you would expect to get a different result if you made another call to end() after the data was changed. But what is confusing is the value of the iterator object you already made changing.

Is there a way to create iterators that allow you to work on a set range of data in a circular_buffer and do not have their values changed even if additional data is added to the buffer while working on a range?

If not, is there a preferred 'non-iterator' pattern that might apply, or a different container class that might allow this?

#include "stdafx.h"
#include <boost\thread.hpp>
#include <boost\thread\thread_time.hpp>
#include <boost\circular_buffer.hpp>

int _tmain(int argc, _TCHAR* argv[])
{
    boost::circular_buffer<int> buffer(20);

    buffer.push_back(99);
    buffer.push_back(99);
    buffer.push_back(99);

    boost::circular_buffer<int>::const_iterator itBegin = buffer.begin();
    boost::circular_buffer<int>::const_iterator itEnd = buffer.end();

    int count = itEnd - itBegin;
    printf("itEnd - itBegin == %i\n", count); //prints 3

    /* another thread (or this one) pushes an item*/
    buffer.push_back(99);

    /*we check values of begin and end again, they've changed, even though we have not done anything in this thread*/
    count = itEnd - itBegin;
    printf("itEnd - itBegin == %i\n", count); //prints 4 
}

---

Update for more detailed response to ronag I am looking for a producer consumer type model, and the bounded buffer example shown in the boost documentation is CLOSE to what I need. with the following two exceptions.

1. I need to be able to read more than one element of data off the buffer at a time, inspect them, which may not be trivial, and then choose how many items to to remove from the buffer.

   fake example: trying to process the two words hello & world from a string of characters.

   read 1, buffer contains h-e-l, do not remove chars from the buffer. -more producing read 2, buffer contains h-e-l-l-o-w-o, I found 'hello' remove 5 chars, buffer now has w-o -more producing read 3, buffer contains w-o-r-l-d, process 'world', remove another 5 characters.

2. I need to not block the producer thread when reading, unless the buffer is full.

Answer 1

According to the boost::circular_buffer::iterator docs, your iterators should remain valid. (Always the first thing I check when mutating and iterating a container at the same time.) So your example code is legal.

What is happening is due to STL iterator convention: end() doesn't point at an element, but rather to the imaginary one-past-the-last element. After the fourth push_back, the distance from itBegin (the first element) to itEnd (one-past-the-last element) has increased.

One solution may be to hold an iterator pointing to a concrete element, e.g.

itPenultimate = itEnd - 1;

Now the distance from itBegin to itPenultimate will remain the same even when the circular buffer is extended, as long as it's not in the range.