Search code examples
pythonnumpypython-c-api

numpy nd array layout in memory


Transposing an ndarry does not change the actually layout in memory i.e. after transpose the ndarry is not 'C' order anymore

I wrote a cpp extension for numpy tasks thus if the order the dimensions given are not in the order the cpp expects I use np.traspose before calling the extension though the actual memory layout does not change as expected.

Original

x = np.random.rand(1, 10, 10, 6) 
print('shape', x.shape, 'strides', x.strides) 

output: shape (1, 10, 10, 6) strides (4800, 480, 48, 8)

now traspose data expect shape to transpose but strides to still be in decrementing order

x_t = np.transpose(x, [0, 3, 1, 2]) 
print('shape', x_t.shape, 'strides', x_t.strides) 

shape (1, 6, 10, 10) strides (4800, 8, 480, 48)

i.e. data layout stayed the same in memory

I would to transpose to change the actual data memory layout for efficient data looping


Solution

  • one method i found is to use np.copy as such:

    x_t = np.transpose(x, [0, 3, 1, 2]).copy(order='C') 
    print('shape', x_t.shape, 'strides', x_t.strides) 
    

    shape (1, 6, 10, 10) strides (4800, 800, 80, 8)