I want to create an array from a compressed masked array and a corresponding mask. Its easier to explain this with an example:
>>> x=np.ma.array(np.arange(4).reshape((2,2)), mask = [[True,True],[False,False]])
>>> y=x.compressed()
>>> y
array([ 2, 3])
Now I want to create an array in the same shape as x where the masked values get a standard value (for example -1) and the rest is filled up with a given array. It should work like this:
>>> z = decompress(y, mask=[[True,True],[False,False]], default=-1)
>>> z
array([[-1, -1],
[ 2, 3]])
The question is: Is there any method like "decompress", or do i need to code it myself? In Fortran this is done by the methods "pack" and "unpack". Thanks for any suggestions.
While I've answered a number of ma questions I'm by no means an expert with it. But I'll explore the issue
Let's generalize your a array a bit:
In [934]: x=np.ma.array(np.arange(6).reshape((2,3)), mask = [[True,True,False],[False,False,True]])
In [935]: x
Out[935]:
masked_array(data =
[[-- -- 2]
[3 4 --]],
mask =
[[ True True False]
[False False True]],
fill_value = 999999)
In [936]: y=x.compressed()
In [937]: y
Out[937]: array([2, 3, 4])
y has no information about x except a subset of values. Note it is 1d
x stores its values in 2 arrays (actually these are properties that access underlying ._data, ._mask attributes):
In [938]: x.data
Out[938]:
array([[0, 1, 2],
[3, 4, 5]])
In [939]: x.mask
Out[939]:
array([[ True, True, False],
[False, False, True]], dtype=bool)
My guess is that to de-compress we need to make a empty masked array with the correct dtype, shape and mask, and copy the values of y into its data. But what values should be put into the masked elements of data?
Or another way to put the problem - is it possible to copy values from y back onto x?
A possible solution is to copy the new values to x[~x.mask]:
In [957]: z=2*y
In [958]: z
Out[958]: array([4, 6, 8])
In [959]: x[~x.mask]=z
In [960]: x
Out[960]:
masked_array(data =
[[-- -- 4]
[6 8 --]],
mask =
[[ True True False]
[False False True]],
fill_value = 999999)
In [961]: x.data
Out[961]:
array([[0, 1, 4],
[6, 8, 5]])
Or to make a new array
In [975]: w=np.zeros_like(x)
In [976]: w[~w.mask]=y
In [977]: w
Out[977]:
masked_array(data =
[[-- -- 2]
[3 4 --]],
mask =
[[ True True False]
[False False True]],
fill_value = 999999)
In [978]: w.data
Out[978]:
array([[0, 0, 2],
[3, 4, 0]])
Another approach is to make a regular array, full with the invalid values, copy y in like this, and turn the whole thing into a masked array. It's possible that there is a masked array constructor that lets you specify the valid values only along with the mask. But I'd have to dig into the docs for that.
===============
Another sequence of operations that will do this, using np.place for set values
In [1011]: w=np.empty_like(x)
In [1014]: np.place(w,w.mask,999)
In [1015]: np.place(w,~w.mask,[1,2,3])
In [1016]: w
Out[1016]:
masked_array(data =
[[-- -- 1]
[2 3 --]],
mask =
[[ True True False]
[False False True]],
fill_value = 999999)
In [1017]: w.data
Out[1017]:
array([[999, 999, 1],
[ 2, 3, 999]])
====================
Look at
https://github.com/numpy/numpy/blob/master/numpy/ma/core.py
class _MaskedBinaryOperation:
This class is used to implement masked ufunc. It evaluates the ufunc at valid cells (non-masked) and returns a new masked array with the valid ones, leaving the masked values unchanged (from the original)
For example with a simple masked array, +1 does not changed the masked value.
In [1109]: z=np.ma.masked_equal([1,0,2],0)
In [1110]: z
Out[1110]:
masked_array(data = [1 -- 2],
mask = [False True False],
fill_value = 0)
In [1111]: z.data
Out[1111]: array([1, 0, 2])
In [1112]: z+1
Out[1112]:
masked_array(data = [2 -- 3],
mask = [False True False],
fill_value = 0)
In [1113]: _.data
Out[1113]: array([2, 0, 3])
In [1114]: z.compressed()+1
Out[1114]: array([2, 3])
_MaskedUnaryOperation might be simpler to follow, since it only has to work with 1 masked array.
Example, regular log has problems with the masked 0 value:
In [1115]: z.log()
...
/usr/local/bin/ipython3:1: RuntimeWarning: divide by zero encountered in log
#!/usr/bin/python3
Out[1116]:
masked_array(data = [0.0 -- 0.6931471805599453],
mask = [False True False],
fill_value = 0)
but the masked log skips the masked entry:
In [1117]: np.ma.log(z)
Out[1117]:
masked_array(data = [0.0 -- 0.6931471805599453],
mask = [False True False],
fill_value = 0)
In [1118]: _.data
Out[1118]: array([ 0. , 0. , 0.69314718])
oops - _MaskedUnaryOperation might not be that useful. It evaluates the ufunc at all values np.ma.getdata(z), with a errstate context to block warnings. It then uses the mask to copy masked values on to the result (np.copyto(result, d, where=m)).