Search code examples
pythonlistnested-lists

Find indices of nan elements in nested lists and remove them


names=[['Pat','Sam', np.nan, 'Tom', ''], ["Angela", np.nan, "James", ".", "Jackie"]]
values=[[1, 9, 1, 2, 1], [1, 3, 1, 5, 10]]

I have 2 lists: names and values. Each value goes with a name, i.e., Pat corresponds to the value 1 and Sam corresponds to the value 9.

I would like to remove the nan from names and the corresponding values from values.

That is, I want a new_names list that looks like this:

[['Pat','Sam', 'Tom', ''], ["Angela", "James", ".", "Jackie"]]

and a new_values list that looks like this:

[[1, 9, 2, 1], [1, 1, 5, 10]]

My attempt was to first find the indices of these nan entries:

all_nan_idx = []
for idx, name in enumerate(names):
  if pd.isnull(name):
  all_nan_idx.append(idx)

However, the above does not account nested lists.


Solution

  • Simply this?

    import numpy as np
    import pandas as pd
    
    names=[['Pat','Sam', np.nan, 'Tom', ''], ["Angela", np.nan, "James", ".", "Jackie"]]
    values=[[1, 9, 1, 2, 1], [1, 3, 1, 5, 10]]
    
    new_names = []
    new_values = []
    for names_, values_ in zip(names, values):
        n = []
        v = []
        for name, value in zip(names_, values_):
            if not pd.isnull(name):
                n.append(name)
                v.append(value)
        new_names.append(n)
        new_values.append(v)