Search code examples
pythonmultithreadingthread-safety

Partially deleting elements from a bytearray while adding to it from another thread

I have bytearray object, two threads are accessing it, a producer and consumer.

The producer writes to the bytearray: barr.extend(data)

The consumer reads from the bytearray: barr.find(...), barr[x:y]

The producer adds alot of data to the bytearray so I want to free up memory occasionally by del barr[:last_read_pos]

Is this operation thread safe?

Thanks.

Solution

  • "Is this operation thread safe?" "yes", but only due to the GIL (Global Interpreter Lock). It is actually unsafe up to the point that although it would work, it should not be done. Just use a Lock - it is painless enough.

    Or, a queue, which is built for this: adding byte-object as Queue items, instead of extending a bytearray with then make your code at one time simpler and more efficient, as extending a bytearray involves copying the bytes around, realocating the whole array at times, and so on.

    So - reading more carefully your requirements in the comments: since you want the continuous byte structure, a simple thing to do is to use the Queue to post bytes objects on the producer side, and assemble your continuous buffer on the consumer side.