How to stream data in KDB?

Question

I have access to a realtime KDB server that has tables with new data arriving every millisecond.

Currently, I'm just using a naive method which is basically like:

.z.ts:{
    newData: getNewData[];   / get data arriving in the last second
    data::data uj newData;
};
\t 100;

to ensure that my data (named data) is constantly updated.

However, the uj is very slow (probably due to constant reallocation of memory) and polling is just plain awkward.

I've heard KDB is meant to be good at handling this kind of streaming tick data, so is there a better way? Perhaps some push-based method without need for uj?

Answer 1

Rather than polling. Use kdb+tick the publish subscriber architecture for kdb+.

• Official Manual: https://github.com/KxSystems/kdb/blob/master/d/tick.htm
• kdb+ Tick overview: http://www.timestored.com/kdb-guides/kdb-tick-data-store
• Source code: https://github.com/KxSystems/kdb-tick