Search code examples
pandascovariancefinanceportfolio

Pandas: apply function pairwise to a dataframe and a panel


In the context of finance, suppose there is a dataframe of asset weights and a panel of daily covariance matrix:

w = pd.DataFrame({'Date':pd.to_datetime(['2016-01-01','2016-01-02','2016-01-03']),'A1':[0.3,0.1,0.1],'A2':[0.4,0.4,0.4]}).set_index(['Date'])
covar = [[[0.000087,0.000017],[0.000087,0.000017],[0.000087,0.000017]],[[0.000017,0.00019],[0.000017,0.00019],[0.000017,0.00019]]]
covPanel = pd.Panel(covar, items=['A1', 'A2'],  major_axis=pd.to_datetime(['2016-01-01','2016-01-02','2016-01-03']), minor_axis=['A1', 'A2'])

To calculate the 1-day portfolio variance, the below function can be used:

def portVar(w,sigma):
    return w.dot(sigma.dot(w))

I can apply the last row of weights to the covariance matrix each day to get daily variance:

out = covPanel.apply(lambda cov1: portVar(w.iloc[-1,:],cov1),axis = [2,0])

But how do I apply the above function to the dataframe and covariance matrix pairwise, for every day (without loop)?

In other words, something like:

pd.ApplyPairwise(portVar,w,covPanel) 

and return the daily variance just as "out" above?


Solution

  • option 1
    rewrite portVar

    pass entire panel to the function being applied and use xs to get the appropriate cross section for that particular date's weight. The date is in the name attribute.


    def portVar(w, sigma):
        s = sigma.xs(w.name, axis='major')
        return w.dot(s.dot(w))
    
    w.apply(portVar, 1, sigma=covPanel)
    
    Date
    2016-01-01    0.000042
    2016-01-02    0.000033
    2016-01-03    0.000033
    dtype: float64
    

    option 2
    numpy broadcasting

    cv = covPanel.values
    wv = w.values
    
    pd.Series(((wv[None, :] * cv).sum(-1).T * wv).sum(1), w.index)
    
    Date
    2016-01-01    0.000042
    2016-01-02    0.000033
    2016-01-03    0.000033
    dtype: float64
    

    response to comment

    A generic python way to do things pair wise is zip
    I'll use a list comprehension to produce a the list we're looking for. Note the transpose on the panel object to ensure the dates are the first dimension.

    def portVar(w,sigma):
        return w.dot(sigma.dot(w))
    
    [portVar(w_, s_) for w_, s_ in zip(w.values, covPanel.transpose(1, 0, 2).values)]