Search code examples
pythonarraysnumpypyopencl

Converting numpy Void array to record array?


I have a numpy array with 8 16 byte long void records that looks like this:

array([[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
       [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]], 
       dtype='|V16')

And need to cast it to an array with 8 16B long records of a custom dtype that looks like this:

[(('x', 's0'), '<u4'), (('y', 's1'), '<u4'), (('z', 's2'), '<u4'), ('padding0', '<u4')]

How can I achieve that?

I've tried array.astype(self.dtype, copy=False, casting="unsafe"), but I get

ValueError: setting an array element with a sequence.

Which doesn't make much sense to me.

This data comes from PyOpenCL (memory mapping a buffer), I can't really change the input format or the dtype.


Solution

  • As long as the number of bytes match, view can make this kind of transformation. It just changes how the data buffer is 'viewed'.

    In [36]: dt=np.dtype([(('x', 's0'), '<u4'), (('y', 's1'), '<u4'), (('z', 's2'), '<u4'), ('padding0', '<u4')])
    In [37]: dt
    Out[37]: dtype([(('x', 's0'), '<u4'), (('y', 's1'), '<u4'), (('z', 's2'), '<u4'), ('padding0', '<u4')])
    
    In [39]: x = np.zeros((3,), dtype=dt)
    In [40]: x
    Out[40]: 
    array([(0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0)],
          dtype=[(('x', 's0'), '<u4'), (('y', 's1'), '<u4'), (('z', 's2'), '<u4'), ('padding0', '<u4')])
    In [41]: x.view('|V16')
    Out[41]: 
    array([[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0],
           [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0], [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]],
          dtype='|V16')
    In [42]: x.view('|V16').view(dt)
    Out[42]: 
    array([(0, 0, 0, 0), (0, 0, 0, 0), (0, 0, 0, 0)],
          dtype=[(('x', 's0'), '<u4'), (('y', 's1'), '<u4'), (('z', 's2'), '<u4'), ('padding0', '<u4')])
    

    I often have to experiment to figure out whether astype or view is the right way to transform a structured array.