Search code examples
pythonpython-itertoolsnumpy-ndarray

fill missing values in 3D list with zeros to create 3D numpy array


I have a 3D list ll which can be of size 100 K * 10 * 3

ll = [
[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10,11,12]], [[6, 7, 8],[12, 13, 14]], [[10, 20, 30], [40, 50, 60], [70, 80, 90]]
]

I want it to be

ll = [[[1, 2, 3], [4, 5, 6], [7, 8, 9], [10,11,12]], [[6, 7, 8],[12, 13, 14], [0, 0, 0], [0, 0, 0]], [[10, 20, 30], [40, 50, 60], [70, 80, 90], [0,0,0]]]

so that I can create a1 = np.array(l1)

a1

array([
[[1,2,3], [4,5,6], [7,8,9], [10,11,12]]
[[6,7,8], [12,13,14], [0,0,0], [0,0,0]]
[[10, 20, 30], [40, 50, 60], [70, 80, 90], [0,0,0]]
])

I have read the following but they are for 2D, i am not able to do it for 3D.

https://stackoverflow.com/a/38619333/5202279

https://stackoverflow.com/a/43149308/5202279


Solution

  • Here's a way that allocates the NumPy array up front then copies the data over. Assuming you don't actually need the expanded ll, this should use less memory than appending the 0-triples to ll before creating a1:

    a1 = np.zeros((len(ll), max([len(k) for k in ll]), 3))
    for ctr,k in enumerate(ll):
         a1[ctr,:len(k),:] = k
    
    a1
    array([[[ 1.,  2.,  3.],
            [ 4.,  5.,  6.],
            [ 7.,  8.,  9.],
            [10., 11., 12.]],
    
           [[ 6.,  7.,  8.],
            [12., 13., 14.],
            [ 0.,  0.,  0.],
            [ 0.,  0.,  0.]],
    
           [[10., 20., 30.],
            [40., 50., 60.],
            [70., 80., 90.],
            [ 0.,  0.,  0.]]])
    

    max([len(k) for k in ll]) tells us the maximum number of triples in any member of ll. We allocate a 0-initialized NumPy array of the desired size. Then in the loop, smart indexing tells us where in a1 to copy each member of ll.