In Python I have a large xarray dataarray (da) of the shape
In [1]: da.shape
Out[1]: (744, 24, 30, 131, 215)
I need to perform the operation da = 10**da however, I'm running into a memory error
MemoryError: Unable to allocate 56.2 GiB for an array with shape (744, 24, 30, 131, 215) and data type float32
Is there a way to do the operation iteratively? I tried to manually loop but that gives the same error.
for ii in range(len(da[:,0,0,0,0])):
da[ii,:,:,:,:] = 10**da[ii,:,:,:,:]
It works fine with smaller arrays e.g., (24, 24, 30, 131, 215).
Chunking the dataarray did the trick.
da = da.chunk({'dim0':50,'dim1': 24, 'dim2': 30, 'dim3': 131, 'dim4': 215})
da = 10**da