Search code examples
pythonnumpystructured-arraymasked-array

How can I mask elements of a record array in Numpy?


I understand how to create a masked array, and I would like to use masking in a record array so that I can access this data using named attributes. The masking seems to be "lost" when I create a record array from a masked array:

>>> data = np.ma.array(np.ma.zeros(30, dtype=[('date', '|O4'), ('price', '<f8')]),mask=[i<10 for i in range(30)])
>>> data
masked_array(data = [(--, --) (--, --) (--, --) (--, --) (--, --) (--, --) (--, --) (--, --)
(--, --) (--, --) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0)
(0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0)],
         mask = [(True, True) (True, True) (True, True) (True, True) (True, True)
(True, True) (True, True) (True, True) (True, True) (True, True)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)],
   fill_value = ('?', 1e+20),
        dtype = [('date', '|O4'), ('price', '<f8')])

>>> r = data.view(np.recarray)
>>> r
rec.array([(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
           (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
           (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
           (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
           (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0)], 
           dtype=[('date', '|O4'), ('price', '<f8')])

When I access a record the data is not masked:

>>> r.date[0]
0

Unlike in the original array:

>>> data['date'][0]
masked_array(data = --,
             mask = True,
       fill_value = 1e+20)

       fill_value = 1e+20)

What can I do? Does the record array not support masking? Browsing on the web I have seen some code examples that seem to suggest otherwise, but it wasn't very clear. Hoping I can get a good answer here.


Solution

  • I haven't found much documentation on numpy.ma.mrecords.MaskedRecords, except for a brief mention here. You can find some examples on how to use it by studying the unit tests that come with numpy. (e.g. /usr/lib/python2.6/dist-packages/numpy/ma/tests/test_mrecords.py).

    import numpy as np
    import numpy.ma.mrecords as mrecords
    
    data = np.ma.array(
        np.ma.zeros(30, dtype=[('date', '|O4'), ('price', '<f8')]),
        mask=[i<10 for i in range(30)])
    
    r = data.view(mrecords.mrecarray)
    
    print(r.date[0])
    # --