Search code examples
pandasformattingcovariance

Pandas covariance returning zero


I'm having trouble using the cov function for a pandas dataframe. Can someone explain why the first version is erroneously returning zeros?

In[2]:    temp.cov()
Out[2]: 
      dogs  cats
dogs   0.0   0.0
cats   0.0   0.0

In[3]:    temp.iloc[1:,:].cov()
Out[3]: 
            dogs       cats
dogs  271.813913  49.549741
cats   49.549741  11.500842

The dataframe is below...

         dogs      cats
0   -7.130125 -0.042310
1  -12.302285 -1.852217
2  -20.583190 -3.504634
3   -6.589786 -1.631955
4   19.512195  3.250522
5   18.612521  3.459353
6    7.029877 -1.525133
7   23.172906  3.555051
8   18.691589  1.913509
9  -27.184466 -7.424187
10  31.307551  6.743372
11  13.628620 -1.604586
12   8.756567  3.338248
13  -3.565062 -0.082786
14   3.539823  0.081160
15  -1.782531 -0.027595
16 -12.433393 -1.806514

{'dogs': {0: -7.1301247771835552, 1: -12.302284710017597, 2: -20.583190394511174, 3: -6.5897858319604596, 4: 19.51219512195118, 5: 18.612521150592254, 6: 7.0298769771528491, 7: 23.172905525846723, 8: 18.691588785046733, 9: -27.184466019417485, 10: 31.307550644567193, 11: 13.628620102214661, 12: 8.7565674255691839, 13: -3.5650623885917776, 14: 3.5398230088495297, 15: -1.7825311942958333, 16: -12.433392539964562}, 'cats': {0: -0.04231002139298834, 1: -1.8522167487684538, 2: -3.5046343844700178, 3: -1.6319546791088158, 4: 3.2505224053865356, 5: 3.4593526068693037, 6: -1.5251327504292034, 7: 3.5550505255468323, 8: 1.9135093761959987, 9: -7.424187066130683, 10: 6.7433720685594061, 11: -1.604586062335045, 12: 3.3382478256752668, 13: -0.082786254170352791, 14: 0.081160267179636314, 15: -0.027595418056747256, 16: -1.8065138668443836}}

also, if someone could show me an easy way to format data like this so that others could easily use it, I would appreciate it / will update it.


Solution

  • This appears to be a known issue with numpy and a recent version MKL. Running update conda --all seems to fix my issue.

    It's well documented here... https://github.com/numpy/numpy/issues/9758