Search code examples
pythonarraysinitializationcythonmemoryview

Initialise Cython Memoryview efficiently


I'm currently setting my MemoryViews in my Cython pyx file as follows:

@cython.boundscheck(False)
cdef int[:] fill_memview():
    # This happens inside a big loop so needs to be fast
    cdef int[:] x = np.empty(10)
    for i in range(10):
        x[i] = i
    return x

cdef stupid_loop():
    for i in range(10000):
        fill_memview()

When I compile the pyx file with cython -a foo.pyx the line cdef int[:] x = np.empty(10) shows in the resulting annotated html file in dark yellow (meaning it has lots of Python calls slowing things down.)

How can I instatiate my typed Memoryview better?


Solution

  • See this answer for a comparison of different ways of allocating memory. If your needs are simple (just indexing) pay particular attention to 'cpython.array raw C type', you can create a cpython array for fast creation then use as_ints[i] for fast unsafe indexing, or if you really do need a memory view, the memory view on a cpython array is 3x faster than a numpy array.

    Without having a bigger picture of what your code does it's difficult to offer more specific advice. For example if possible you would be better served using a two-dimensional array as it tends to be much more efficient to allocate one large chunk of memory than lots of little ones, and for instance it's much quicker to make lots of little memory view slices of one big memory view with a big hunk of allocated memory, than to create a bunch of little memory views each with their own little piece of allocated memory.