Search code examples
precisionmatrix-multiplicationkdbq-lang

q kdb matrix multiplication precision


I am doing the matrix multiplication / dot product of two vectors, but losing significant precision in the process. My two tables are:

A:flip (`WORLD`MOMENTUM`VOLATIL`VALUE`SIZE`SIZENONL`GROWTH`LIQUID`LEVERAGE`ENERGY`OILGAS`OILEXPL`CHEMICAL`CONSTPP`DIVMETAL`PRECMETL`STEEL`CAPGOODS`COMMSVCS`TRANSPRT`AIRLINES`AUTOCOMP`CONSDUR`CONSVCS`MEDIA`RETAIL`FOODRETL`FOODPRD`HSHLDPRD`HEALTH`BIOTECH`PHARMAC`BANKS`DIVFINAN`INSURAN`REALEST`INTERNET`SOFTWARE`COMMUNIC`COMPUTER`SEMICOND`TELECOM`UTILITY`ARE`ARG`AUS`AUT`BEL`BHR`BRA`CAN`CHE`CHL`CHN`CHX`COL`CZE`DEU`DNK`EGY`ESP`FIN`FRA`GBR`GRC`HKG`HUN`IDN`IND`IRL`ISR`ITA`JOR`JPN`KOR`KWT`MAR`MEX`MYS`NLD`NOR`NZL`OMN`PAK`PER`PHL`POL`PRT`QAT`RUS`SAU`SGP`SWE`THA`TUR`TWN`USA`ZAF`AREC`ARGC`AUSC`AUTC`BELC`BHRC`BRAC`CANC`CHEC`CHLC`CHNC`COLC`CZEC`DEUC`DNKC`EGYC`EMUC`ESPC`FINC`FRAC`GBRC`GRCC`HKGC`HUNC`IDNC`INDC`IRLC`ISRC`ITAC`JORC`JPNC`KORC`KWTC`MARC`MEXC`MYSC`NLDC`NORC`NZLC`OMNC`PAKC`PERC`PHLC`POLC`PRTC`QATC`RUSC`SAUC`SGPC`SWEC`THAC`TURC`TWNC`USAC`ZAFC)!(enlist 610216.7;enlist -167790.7;enlist -61640.11;enlist 1211006f;enlist -805318.3;enlist -144186.8;enlist 484780.3;enlist 729160.8;enlist -1223112f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist -63589.8;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 47259.38;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist -42577.22;enlist 0f;enlist 721972.9;enlist 0f;enlist -32170.51;enlist 0f;enlist 0f;enlist 0f;enlist -152106.2;enlist 131428.1;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 610216.7;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 0f;enlist 610216.7;enlist 0f;enlist 0f;enlist 0f);
B:flip (enlist `WORLD)!(enlist 48.29499 -5.301547 17.17246 2.146016 1.69432 0.295386 1.296268 1.381681 1.931021 7.972107 1.824557 10.18463 6.161642 4.141302 20.12705 -44.44252 22.11896 6.354314 -0.455013 -1.28214 -5.993917 9.041717 -1.749305 -5.074565 -2.914975 -4.31613 -9.272181 -11.28901 -8.72011 -7.808923 -0.938631 -8.579992 6.898276 8.94749 6.497003 -1.967439 -2.090085 -0.254925 1.320048 5.30489 5.746626 -8.236259 -14.77134 -9.308569 -14.18904 -7.501979 3.634038 -3.201371 -32.09656 -5.156152 -8.858147 0.763125 -20.76443 -16.71714 -2.192779 -22.27704 -5.303177 3.503334 -2.870094 -1.125126 6.617578 -1.169103 7.511715 -4.384823 10.17381 0.769835 -3.488056 -18.70038 -6.739745 -4.994236 -7.670161 7.711241 -29.75733 5.311001 -22.09107 -33.23776 -38.32805 -17.19578 -14.23223 6.033368 -2.48355 -7.678366 -17.67733 -27.92534 -15.13988 -11.78941 -11.99404 -0.087656 -12.29577 -6.127644 -8.919363 2.165941 -0.196319 -9.859482 -6.289163 -12.44774 3.835522 -11.39685 6.8e-005 10.45363 30.1068 3.294173 3.294173 0.11509 33.12787 25.92562 -1.497061 20.91031 3.470695 32.76403 5.528537 3.294173 3.31778 -0.172639 3.294173 3.294173 3.294173 3.294173 13.11109 3.294173 0.708191 14.87056 8.111332 11.23322 3.294173 10.9295 3.294173 0.056315 -25.3915 26.04028 -0.344381 3.045744 41.80897 15.30789 3.294173 18.46745 24.13028 0.065611 0.511451 7.17594 9.577834 19.91554 3.294173 0.448114 33.1457 0.036037 9.321848 13.95029 7.793591 30.57135 11.67946 0 44.67686);

I have tried a few combinations of mmu and $, including:

(raze value flip A) mmu flip value flip B
(flip value flip A) mmu flip value flip B

The result I get is 41,756,840, the correct result is 41,756,847. I cannot see where the problem is, any ideas welcome, thank you.


Solution

  • I’ve failed to reproduce your problem. I’m using V3.6 32-bit on macOS Sierra.

    q)\P 10
    q)A
    WORLD    MOMENTUM  VOLATIL   VALUE   SIZE      SIZENONL  GROWTH   LIQUID   LE..
    -----------------------------------------------------------------------------..
    610216.7 -167790.7 -61640.11 1211006 -805318.3 -144186.8 484780.3 729160.8 -1..
    q)3#B
    WORLD
    ---------
    48.29499
    -5.301547
    17.17246
    q)(raze value flip A) mmu flip value flip B
    ,41756847.3
    q)(flip value flip A) mmu flip value flip B
    41756847.3
    

    You don’t of course need mmu here. You have two vectors and want the sum of their products.

    q)a:610216.7 -167790.7 -61640.11 1211006 -805318.3 -144186.8 484780.3 729160.8 -1223112 0 0 0 0 -63589.8 0 0 0 0 0 0 0 0 47259.38 0 0 0 0 0 0 0 0 0 0 -42577.22 0 721972.9 0 -32170.51 0 0 0 -152106.2 131428.1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 610216.7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 610216.7 0 0 0
    q)b:48.29499 -5.301547 17.17246 2.146016 1.69432 0.295386 1.296268 1.381681 1.931021 7.972107 1.824557 10.18463 6.161642 4.141302 20.12705 -44.44252 22.11896 6.354314 -0.455013 -1.28214 -5.993917 9.041717 -1.749305 -5.074565 -2.914975 -4.31613 -9.272181 -11.28901 -8.72011 -7.808923 -0.938631 -8.579992 6.898276 8.94749 6.497003 -1.967439 -2.090085 -0.254925 1.320048 5.30489 5.746626 -8.236259 -14.77134 -9.308569 -14.18904 -7.501979 3.634038 -3.201371 -32.09656 -5.156152 -8.858147 0.763125 -20.76443 -16.71714 -2.192779 -22.27704 -5.303177 3.503334 -2.870094 -1.125126 6.617578 -1.169103 7.511715 -4.384823 10.17381 0.769835 -3.488056 -18.70038 -6.739745 -4.994236 -7.670161 7.711241 -29.75733 5.311001 -22.09107 -33.23776 -38.32805 -17.19578 -14.23223 6.033368 -2.48355 -7.678366 -17.67733 -27.92534 -15.13988 -11.78941 -11.99404 -0.087656 -12.29577 -6.127644 -8.919363 2.165941 -0.196319 -9.859482 -6.289163 -12.44774 3.835522 -11.39685 6.8e-005 10.45363 30.1068 3.294173 3.294173 0.11509 33.12787 25.92562 -1.497061 20.91031 3.470695 32.76403 5.528537 3.294173 3.31778 -0.172639 3.294173 3.294173 3.294173 3.294173 13.11109 3.294173 0.708191 14.87056 8.111332 11.23322 3.294173 10.9295 3.294173 0.056315 -25.3915 26.04028 -0.344381 3.045744 41.80897 15.30789 3.294173 18.46745 24.13028 0.065611 0.511451 7.17594 9.577834 19.91554 3.294173 0.448114 33.1457 0.036037 9.321848 13.95029 7.793591 30.57135 11.67946 0 44.67686
    q)sum a*b
    41756847.3
    

    A possible source of trouble is the location of backticks in the symbol vector in A, which can’t be seen directly in your posted question.