Search code examples
pythonpandasdataframenumpyrolling-computation

std composed with rolling and shift is not working on pandas (Python)

Consider the following dataframe

df = pd.DataFrame()
df['Amount'] = [13,17,31,48]

I want to calculate for each row the std of the previous 2 values of the column "Amount". For example:

  • For the third row, the value should be the std of 17 and 13 (which is 2).
  • For the fourth row, the value should be the std of 31 and 17 (which is 7).

This is what I did:

df['std previous 2 weeks'] = df['Amount'].shift(1).rolling(2).std()

But this is not working. I thought that my problem was an index problem. But this works perfectly with the sum method.

df['total amount of previous 2 weeks'] = df['Amount'].shift(1).rolling(2).sum()

PD : I know that this can be done in some other ways but I want to know the reason for why this does not work (and how to fix it).

Solution

  • You could shift after rolling.std. Also the degrees of freedom is 1 by default, it seems you want it to be 0.

    df['Stdev'] = df['Amount'].rolling(2).std(ddof=0).shift()

    Output:

       Amount  Stdev
0      13    NaN
1      17    NaN
2      31    2.0
3      48    7.0