Python : can I get a memoryview or a bytearray mapping to a mmap

I'm writing a fairly simple python program that reads a mmap area, and perhaps modifies some bytes in that mmap area, or perhaps takes a copy of some of these bytes.

The mmap area is some MB big, and split in frames 2048 bytes big. The program loops very quickly thru that area continuously, and sometimes makes a copy, almost all the time touches a few bytes in each of the 2048 bytes frames.

To limit time spent in object creation to the minimum (ie:only create object when needing a copy), I want to create a list of "some object" pointing to the memory for each 2048 bytes frames.

So far I've managed to do that with ctypes (array of 2048 char), but I find that cumbersome to use. I'm wondering if there is a way to create a memoryview, or a bytearray-like object from the mmap object, while not making an actual copy of the memory, but rather that the object just points at the mmap.

Here is some code:

from ctypes import *
import mmap

class fixed_size_frame_2048(Structure):
    _fields_ = [("data",c_ubyte * 2048)]

nb_frames = 4096
#mmap initializing code resulting in a map object pointing to shared memory
#the map object is nb_frames*2048 bytes big
map = mmap.mmap(
            -1,
            2048 * nb_frames,
            flags=(mmap.MAP_SHARED),
            prot=(mmap.PROT_READ | mmap.PROT_WRITE)
            )

.....

#initial creation of frame objects pointing into the mmap area
frames= [ None ] * nb_frames
    for i in range(nb_frames) :
        frames[i] = fixed_size_frame_2048.from_buffer(self.map,2048*i)
        #that I don't manage to make work
        #self.frames[i] = memoryview(self.map[2048*i : 2048*(i+1)])

#in the following loop, I never create an object unless I need a copy
position = 0
while True :
    if frames[position].data[2] != 0 :
        #copy the frame to another object
        newcopy = bytearray(frames[position].data)
        #do some other stuff with the new object
        #....


    #write a byte or a few into the mmap area (I'm looking for a more pythonic thing than ctypes here)
    frames[position].data[2] = 0

    #loop stuff
    position += 1
    position %= nb_frames

Looking forward to read any answers, I'm rather stumped by that one. I must be missing something obvious, but what ?

It was rather late last night, so take the following with a grain of salt: it seems memoryview returns me the address of the ctype object, not the memory area pointed to by the ctype object. And if I run bytearray() on a map[:] slice, I get a copy, and if I try memoryview on the slice, it seems to insist in pointing to the start of the mmap. (or at least: it points always to the same place in the mmap area for all 4096 frames -).

Solution

It's crucial to realize that slice of mmap results in bytes. That means the data is copied and hence is no longer connected with the original mmap.

So at first it's necessary to actually create memoryview out of mmap itself and slice the resulting memoryview afterwards. This way you get a view to the memory provided by mmap.

The same logic applies when creating memoryview of part of bytearray, array, etc.

So the code would be:

frames = [ None ] * nb_frames
for i in range(nb_frames) :
    frames[i] = memoryview(map)[2048*i : 2048*(i+1)]

or IMHO more pythonic:

frames = [memoryview(map)[2048*i : 2048*(i+1)] for i in range(nb_frames)]