Search code examples
pythonarrayskdb

KDB embedPy use array from Q in Python

I am using JupyterQ and have two 2 tables in q:

tbl1: ([]month:(1997.01;1997.01;1997.01);
    ric:(3#`GTGS.SI);
    date:(1997.01.06T01:19:21.160;1997.01.06T01:53:57.230;1997.01.06T07:24:19.240);
    price:(0.71;0.72;0.73));

tbl2: ([]month:(1997.01;1997.01;1997.01);
    ric:(3#`NSWF.SI);
    date:(1997.01.06T01:19:21.160;1997.01.06T01:53:57.230;1997.01.06T07:24:19.240);
    price:(0.45;0.46;0.47));

I have a class teCalc defined in python.

I want to pass to it two arrays as follows (in python): teCalc.addObservations(tbl1, tbl2)

How do I pass those q tables to the class in python?

The following function:

print .p.import[`numpy;`:array][tbl1]

gives me:

{'month': array([1997.01, 1997.01, 1997.01]), 'ric': ['GTGS.SI', 'GTGS.SI', 'GTGS.SI'], 'date': array([-1089.94489398, -1089.92086539, -1089.69144398]), 'price': array([0.71, 0.72, 0.73])}

So I can see that it is possible to convert the table in q to a python object.

However, all my attempts to assign it to a variable in python have failed. If I reference either tbl1 or tbl2 in a python line of code in JupyterQ (for example p)print(tbl1)) then the tbl1 variable name isn't recognised.

Please do advise. How do I use data from Q on a class created in Python in JupyterQ?

Thanks and regards,

Simon

Solution

  • [edit]: This should be embedPy not embedQ - my mistake!!

    I got this answer from Connor at KX.

    The choice is import the data to python using Set or import the python class to Q using Get.

    Really cool.

    Simon

    Hi Simon,

    Assigning q objects to be accessible from functions defined in the Python side of the API requires you to set the objects. In the case you're describing where you want to run

    p)print(tbl1)

    You would need to run this in the following steps:

    // Define the q table
mytab:([]5?1f;5?1f)
// set the q table converted to a python dictionary to python variable tbl1
.p.set[`tbl1;mytab]
// Print from the python side of the interface
p)print(tbl1)

    With regards to calling the class with the q objects you would either need to do the sets from q side to python as above .i.e

    .p.set[`tbl1;tbl1]
.p.set[`tbl2;tbl2]
p)teCalc.addObservations(tbl1, tbl2)

    or alternatively 'get' the python callable object and assign it to a q callable function and run directly on the q data without a set

    // Load the script containing the required class
\l myclass.p
// Retrieve the class
teCalc:.p.get[`teCalc]
// Access the relevant function
addObservations:teCalc[`:addObservations]
// Run on the relevant q data and return result to q (denoted by trailing `)
addObservations[tbl1;tbl2]`

    It should be noted that when embedPy converts q tables to Python it does so as a list of dictionaries (this is the accurate representation of what a q table is but may be counter intuitive) as such if you need the data to be a pandas dataframe for you use case you may need to do the conversion manually. This is done using 2 functions .ml.tab2df/.ml.df2tab which are both available within the Kx Machine learning toolkit here.

    More information on embedPy is available at the below links which outline in detail all of the above

    https://code.kx.com/q/ml/embedpy/userguide/

    https://code.kx.com/q/ml/embedpy/faq/

    All the best,

    Conor