Merge 2D list of masked arrays with different lengths

Question

I have a very large list of masked arrays, which I want to combine together, but the arrays have different lengths. To keep it simple this is what I want to do, I want to obtain C:

A=[[--,--,--,...,--]
   [1,2,3,...,--,--],
   ...
B=[[--,--,--,...,--],
   [4,5,6,...,--,--],
   ...
C=A+B
C=[[--,--,--,...,--,--,--,--,...,--],
   [1,2,3,...,--,--,4,5,6,...,--,--],
   ...
len(A)= 81
len(B)= 81
len(A[0])=2700
len(B[0])= 5000

len(C) = 81
len(C[0])= 7700

So I'm basically just putting the two lists next to each other. In reality, my array A looks like this (B is similar):

masked_array(
  data=[[--, --, --, ..., --, --, --],
    [--, --, --, ..., --, --, --],
    [--, --, --, ..., --, --, --],
    ...,
    [-3.6872851848602295, -3.732004165649414, -3.7555367946624756,
     ..., -3.8215177059173584, -3.7747914791107178,
     -3.819281816482544],
    [-3.819749116897583, -3.824702739715576, -3.804812431335449, ...,
     -3.863957643508911, -3.840423345565796, -3.8660500049591064],
    [-3.6894078254699707, -3.7181897163391113, -3.7022457122802734,
     ..., -3.8167803287506104, -3.7095720767974854,
     -3.8254523277282715]],
  mask=[[ True,  True,  True, ...,  True,  True,  True],
    [ True,  True,  True, ...,  True,  True,  True],
    [ True,  True,  True, ...,  True,  True,  True],
    ...,
    [False, False, False, ..., False, False, False],
    [False, False, False, ..., False, False, False],
    [False, False, False, ..., False, False, False]],
  fill_value=9.96921e+36,
  dtype=float32)

The problem is that most commands don't work either because it's a masked array or because it has different sizes. I already checked these questions:

• merge 2D array into a list python
• merging two lists, removing empty strings

Any help is appreciated!

Answer 1

Let's assume we have

import numpy as np
import numpy.ma as ma

# create a 3 x 4 masked array
a = np.arange(12).reshape(3, 4)
a = ma.masked_where(a % 3 == 0, a)
# masked_array(
#  data=[[--, 1, 2, --],
#         [4, 5, --, 7],
#         [8, --, 10, 11]],
#   mask=[[ True, False, False,  True],
#         [False, False,  True, False],
#         [False,  True, False, False]],
#   fill_value=999999)


# create a 3 x 5 masked array
b = np.arange(15).reshape(3,5)
b = ma.masked_where(b % 2 == 0, b)
# masked_array(
#   data=[[--, 1, --, 3, --],
#         [5, --, 7, --, 9],
#         [--, 11, --, 13, --]],
#   mask=[[ True, False,  True, False,  True],
#         [False,  True, False,  True, False],
#         [ True, False,  True, False,  True]],
#   fill_value=999999)

Then you can use numpy.ma.hstack to append these horizontally and preserve the masks:

ma.hstack([a, b])

results in

masked_array(
  data=[[--, 1, 2, --, --, 1, --, 3, --],
        [4, 5, --, 7, 5, --, 7, --, 9],
        [8, --, 10, 11, --, 11, --, 13, --]],
  mask=[[ True, False, False,  True,  True, False,  True, False,  True],
        [False, False,  True, False, False,  True, False,  True, False],
        [False,  True, False, False,  True, False,  True, False,  True]],
  fill_value=999999)