Search code examples
pythonpandasfillna

Pandas fill missing values in dataframe from another dataframe

I cannot find a pandas function (which I had seen before) to substitute the NaN's in a dataframe with values from another dataframe (assuming a common index which can be specified). Any help?

Solution

  • If you have two DataFrames of the same shape, then:

    df[df.isnull()] = d2

    Will do the trick.

    visual representation

    Only locations where df.isnull() evaluates to True (highlighted in green) will be eligible for assignment.

    In practice, the DataFrames aren't always the same size / shape, and transforming methods (especially .shift()) are useful.

    Data coming in is invariably dirty, incomplete, or inconsistent. Par for the course. There's a pretty extensive pandas tutorial and associated cookbook for dealing with these situations.