Search code examples
cythonvoid-pointersreinterpret-casttyped-memory-views

Need to store "untyped" memory view and cast to typed quickly. Is there an reinterpret_cast equivalent in Cython?


Actually I have a working version of what I need, but it is awfully slow:

%%cython -a

cimport numpy as cnp
import numpy as np  # for np.empty

from time import time

cdef packed struct Bar:
    cnp.int64_t dt
    double open
    double high
    double low
    double close
    cnp.int64_t volume

bar_dtype = np.dtype([('dt', 'i8'), ('open', 'f8'), ('high', 'f8'), ('low', 'f8'), ('close', 'f8'), ('volume', 'i8')])

    
cpdef f(Bar bar):
    cdef cnp.ndarray bars_sar
    cdef Bar[:] buffer
    
    bars_sar = np.empty((1000000,), dtype=bar_dtype)
    
    start = time()
    for n in range(1000000):
        buffer = bars_sar
        buffer[0] = bar
        
    print (f'Elapsed: {time() - start}')
    return buffer

def test():
    sar = f(Bar(1,2,3,4,5,6))
    print(sar[0])
    return sar

This 1M iterations loop takes about 3 seconds - because of the line takes memory view from numpy structured array: buffer = bars_sar

The idea is that I need bars_sar untyped. Only when I want to store or read something from it, I want to reinterpret it as a particular type of memory view, and I don't see any problem why it cannot be done fast, but don't know how to do it. I hope, there is something similar to C reinterpret_cast in Cython.

I tried to declare bars_sar as void *, but I'm unable to store memory view address there like:

    cdef Bar[:] view = np.empty((5,), dtype=bar_dtype)
    bars_sar = <void*>view

or even

    cdef Bar[:] view = np.empty((5,), dtype=bar_dtype)
    bars_sar = <void*>&view

The first one results in error C2440 that it cannot convert "__Pyx_memviewslice" to "void *" The second one results in error: "Cannot take address of memoryview slice"

Please suggest


Solution

  • Revisiting my old question, here are 2 ways of achieving what I wanted:

    1. Use Cython.Tempita for creating kinda fused class. (I chose this method)

    2. Get the pointer to raw data (&bars_sar[0]) and work with it directly