Search code examples
node.jskdb+

Save CSV file to kdb+ table


I have a CSV file called candlesData.csv(~300MB). I want to create a new table in kdb+ called candles and have the CSV data be there for faster reading while I use it on my app. I opened a terminal and typed:

show candles:("IJFFFFFJFIFFF"; enlist",") 0: `:path/to/file/candlesData.csv

It shows all the data perfectly. The problem is now how do I store it permanently on the DB. If I exit with \\ and type q again, if I type candles it shows an error.

How do I save this CSV to a table that I can get anytime? and how to access this table afterwards with nodeJS?


Solution

  • You can use the set keyword. If you know what the root of the path is to your database, you can just do

    `:path/to/db/candles set candles
    

    Then when you start q again, if you run

    q path/to/db
    

    it should load in your table

     λ mkdir db
     λ q
    KDB+ 3.6 2019.09.19 Copyright (C) 1993-2019 Kx Systems
    
    q)`:db/table set ([] a: 1 2 3)
    `:db/table
    q)\\
     λ q db
    KDB+ 3.6 2019.09.19 Copyright (C) 1993-2019 Kx Systems
    
    q)table
    a
    -
    1
    2
    3
    q)
    

    In your case, you would run

    q)candles:("IJFFFFFJFIFFF"; enlist",") 0: `:path/to/file/candlesData.csv
    q)`:path/to/db/candles set candles
    

    This is a great resource for this kind of thing:

    https://code.kx.com/q4m3/14_Introduction_to_Kdb+/