Search code examples
pythonpandasdataframenp

Dataframe: compare column value and one row below


I have a dataframe with directions:

        Direction: 
2/01/19 None
1/31/19 Upward
1/30/19 None
1/29/19 None
1/28/19 Downward
1/27/19 None
1/26/19 None
1/25/19 Upward

I want to create a "Momentum" column based on the following conditions (starting at 1/25/19):
1. If the Direction for the corresponding date is 'Upward', then set the value to 'Upward'
2. If the row one below in Momentum is "Upward" set it to 'Upward'
3. If the Direction for the corresponding date is 'Downward', then set it to 'None'
4. Otherwise, set it to 'None'

Said differently, once you reach the 'Upward' status, it should stay that way until you hit 'Downward'

The result should look like:

        Direction:  Momentum:
2/01/19 None        Upward
1/31/19 Upward      Upward
1/30/19 None        None
1/29/19 None        None
1/28/19 Downward    None
1/27/19 None        Upward
1/26/19 None        Upward
1/25/19 Upward      Upward

Is there a way to accomplish this without using loop?


Solution

  • EDITED answer by new data first back fill None values and then replace Downward to Nones:

    #first replace strings Nones to None type
    df['Direction:'] = df['Direction:'].mask(df['Direction:'] == 'None', None)
    
    df['Momentum:'] = df['Direction:'].bfill().mask(lambda x: x == 'Downward', None)
    

    Or:

    s = df['Direction:'].bfill()
    df['Momentum:'] = s.mask(s == 'Downward', None)
    

    print (df)
            Direction:  Momentum:
    2/01/19       None     Upward
    1/31/19     Upward     Upward
    1/30/19       None       None
    1/29/19       None       None
    1/28/19   Downward       None
    1/27/19       None     Upward
    1/26/19       None     Upward
    1/25/19     Upward     Upward
    

    Old answer:

    Use numpy.where with chained boolean mask compared shifted values and also original by | for bitwise OR:

    mask = df['Direction:'].eq('Upward') | df['Direction:'].shift(-1).eq('Upward')
    df['Momentum:'] = np.where(mask, 'Upward', None)
    print (df)
            Direction: Momentum:
    1/31/19       None    Upward
    1/30/19     Upward    Upward
    1/29/19       None      None
    1/28/19       None      None
    1/27/19   Downward      None
    1/26/19       None    Upward
    1/25/19     Upward    Upward