Search code examples
pythonpandasdataframefillna

Change axis for pandas replace ffill

Suppose I have a dataframe that looks like:

df =
       0    1    2
  0  1.0  2.0  3.0
  1  4.0  5.0  NaN
  2  6.0  NaN  NaN

Then it is possible to use df.fillna(method='ffill', axis=1) to obtain:

     0    1    2
0  1.0  2.0  3.0
1  4.0  5.0  5.0
2  6.0  6.0  6.0

i.e. I forward fill the rows. However, now I have a dataframe with -1 instead of np.nan. Pandas has the replace function that also has the possibility to use method='ffill', but replace() does not take an axis argument, so to obtain the same result as above, I would need to call df.T.replace(-1, method='ffill').T. Since transposing is quite expensive (especially considering I'm working on a dataframe of multiple gigabytes), this is not an option. How could I achieve the desired result?

Solution

  • Use mask and ffill

    df.mask(df.eq(-1)).ffill(axis=1)

     0    1    2
0  1.0  2.0  3.0
1  4.0  5.0  5.0
2  6.0  6.0  6.0