Search code examples
kdb

Explanation of code that constructs correlation matrix


I am referring to this answer.

The code to construct a correlation matrix given a table of columns is

u cor/:\:u:flip t where t is a table.

Reading right to left, I understand up till u:flip t. May I please ask for an explanation on what the rest of the code does?

Thanks


Solution

  • If you substitute for a something which gives more visual output, such as join with two vectors, it should be easier to see what the derived function cor/:\: is doing

    q)"123","abc"       // simple join
    "123abc"
    
    q)"123",/:"abc"     // join left arg to each item of right arg
    "123a"
    "123b"
    "123c"
    
    q)"123",/:\:"abc"   // join each item of left arg to each item of right
    "1a" "1b" "1c"
    "2a" "2b" "2c"
    "3a" "3b" "3c"
    

    Back to a simple example of cor

    q)show t:([]a:3?1.0;b:3?1.0;c:3?1.0)
    a         b          c
    -------------------------------
    0.7935513 0.6377554  0.3573039
    0.2037285 0.03845637 0.02547383
    0.7757617 0.8972357  0.688089
    
    q)u cor/:\:u:flip t
     | a         b         c
    -| -----------------------------
    a| 1         0.9474878 0.8529413
    b| 0.9474878 1         0.975085
    c| 0.8529413 0.975085  1
    
    q)show data:value flip t;           // extract the data for clarity
    0.7935513 0.2037285  0.7757617
    0.6377554 0.03845637 0.8972357
    0.3573039 0.02547383 0.688089
    
    q)cor[data 0;]each data             // first row cor each row
    1 0.9474878 0.8529413
    q)cor[data 1;]each data             // second row cor each row
    0.9474878 1 0.975085
    q)cor[data 2;]each data             // last row cor each row
    0.8529413 0.975085 1
    
    q){cor[x]each data}each data        // all at once
    1         0.9474878 0.8529413
    0.9474878 1         0.975085
    0.8529413 0.975085  1
    
    q)data cor/:\:data                  // derived function much nicer
    1         0.9474878 0.8529413
    0.9474878 1         0.975085
    0.8529413 0.975085  1