Search code examples
listkdbq-lang

KDB+ / Q Table creation with dynamic number of columns


I am doing stuff like this:

q)seed:5
q)numvals:3
q)id:`u#1+til seed;
q)vals:numvals cut (seed*numvals)?`8
q)1!([]id;vals)

Now I am getting a table like this:

id| vals                      
--| --------------------------
1 | bhlkdkfd ghmjjepm gphfcddd
2 | jnloahej ipbcbnop heokehhg
3 | eijocfod nbbeopjo ddhiffmp
4 | cndelncn cmcmkckd aelpmccp
5 | nhjdlned cbkgpggg kgbcifbj

Is there a way to automatically unfold the vals list during the table generation to create a table that looks like this:

id| vals0    vals1    vals2              
--| --------------------------
1 | bhlkdkfd ghmjjepm gphfcddd
2 | jnloahej ipbcbnop heokehhg
3 | eijocfod nbbeopjo ddhiffmp
4 | cndelncn cmcmkckd aelpmccp
5 | nhjdlned cbkgpggg kgbcifbj

Solution

  • A table is a dictionary from a dictionary of symbol keys to a list of equal length vectors. Dynamically create the column names using "string til".

    q){ `id xkey update id:i from flip (`$"vals",/:string til y)!(y#x)?\:`8 }[3;4]
    id| vals0    vals1    vals2    vals3
    --| -----------------------------------
    0 | lkamnmmm nfnbfmkm kiblpojl onhlghno
    1 | ehippdmm pocmhfkf edmeodcp iajdgbcc
    2 | kcbjofjk ammebbah dajdafek aafdkpao
    
    q){ `id xkey update id:i from flip (`$"vals",/:string til y)!(y#x)?\:`8 }[3;6]
    id| vals0    vals1    vals2    vals3    vals4    vals5
    --| -----------------------------------------------------
    0 | hkdnolgf jbfokhef medkopgk objjkcmf hcnelcjh mkmiagdh
    1 | kagnelcp lhjkpjen gokacegn iiocipck gpeachbd apmdghdl
    2 | bejncmmp mfhnonen klihfepf oieoflli bbgombmk dkbnlhmd
    

    Your key id column probably isn't necessary, kdb tables have a virtual i column that is the row number. Here I use an update to make it visible.

    To explain this code step by step, start from the right and try evaluating each part.

    A. In kdb we can call a function with two arguments like so: (x is 3, y is 4)

    q){x+y}[3;4]
    7
    

    B. The question mark in kdb can be used for a lot of things, When it has a number on the left and a `8 on the right it generates n random symbols of length 8. Here I use the : adverb to modify the function and call it multiple times for-each item on the left:

    q){(y#x)}[3;4]
    3 3 3 3
    q){(y#x)?\:`8}[3;4]
    gknafbmp odolnkpd pblgicmp
    aphcdfab mgidjeap iledgchk
    ppinbcgh kcijdnbg jleeoccb
    ljjdnami affhgjlm cnijgipc
    

    C.Generate the column names by concatenating strings and again using an adverb:

    q){(`$"vals",/:string til y)}[3;10]
    `vals0`vals1`vals2`vals3`vals4`vals5`vals6`vals7`vals8`vals9
    

    D. A table in kdb is a flipped dictionary. A simple example:

    q)d:`a`b!(1 2 3;4 5 6)
    q)d
    a| 1 2 3
    b| 4 5 6
    q)flip d
    a b
    ---
    1 4
    2 5
    3 6
    

    I use that technique to generate my table:

    q){(`$"vals",/:string til y)!(y#x)?\:`8 }[3;5]
    vals0| djgndbde hlggaadm pbofgnac
    vals1| goppgmfe jlfpmlab ibgkihem
    vals2| npklikob okkmmlbi llnbhgha
    vals3| lbekogce kjclledc cdjbmdpm
    vals4| okmleidn lebjkagh cenblgbi
    q){flip (`$"vals",/:string til y)!(y#x)?\:`8 }[3;5]
    vals0    vals1    vals2    vals3    vals4
    --------------------------------------------
    iiifjhhi idinahfa iejibpcl hebikhmc cjlegpke
    jicockjo kpjphpof kfimnmgh fpjolkmb mkkjldfc
    acjfnnpd ciflaggo ennfgapg bbodkdla bichmhpj
    

    E. Finally update using the virtual i column and use xkey to make it a keyed table:

    q){ `id xkey update id:i from flip (`$"vals",/:string til y)!(y#x)?\:`8 }[3;4]
    id| vals0    vals1    vals2    vals3
    --| -----------------------------------
    0 | mkfcpkdg dnhmifmf gedbdmkb dpdcdhib
    1 | efhafoeh jpidfdno fdbddhgn fimolnmb
    2 | jjmeickp clkbenoe lndodeel pgbfojdb
    

    I advise using this technique to break down any kdb code to try and understand it. Work from right to left, querying the smallest part you don't understand...until you do.