Underlying mechanism of Python's memoryview

Question

Python's memoryview is said to not copy the data upon slicing. Many benchmarks were done, some on stackoverflow, "prooving" this behavior.

While attempting to mess with them, I encountered a weird behavior that I couldn't explain:

>>> arr = bytearray(range(0,15))
>>> mem = memoryview(arr)
>>> mem[5:15] = mem[0:10]
>>> arr
bytearray(b'\x00\x01\x02\x03\x04\x00\x01\x02\x03\x04\x05\x06\x07\x08\t')

On one hand, memoryview "does not" copy the data. On the other hand, this perfectly works!

While I was happy that it "worked", I got saddened by the fact that it works. Well... because it shouldn't.

If Python had a 1 character buffer, the result should have been this:

bytearray(b'\x00\x01\x02\x03\x04\x00\x01\x02\x03\x04\x00\x01\x02\x03\x04')

Basically, upon writing the 5th character, it should have overlapped and read the 1st character that was written earlier. An example for this naive approach:

>>> for i in range(10):
...    m[i+5] = m[i]
>>> a
bytearray(b'\x00\x01\x02\x03\x04\x00\x01\x02\x03\x04\x00\x01\x02\x03\x04')

I tried increasing the memoryview size to large amounts, but it still works, meaning python copies the data in the background, rendering memoryview objects quite pointless.

Is there anywhere I'm wrong in here? Any explanation? How does memoryview work then?

Answer 1

It checks for that:

    if (dptr + size < sptr || sptr + size < dptr)
        memcpy(dptr, sptr, size); /* no overlapping */
    else
        memmove(dptr, sptr, size);

memmove is specified to be safe for overlapping source and destination. How it ensures safety varies from case to case and implementation to implementation, but one technique is to work from right to left instead of left to right if left to right would overwrite not-yet-copied data.