Can a slow consumer kill or slow down tickerplant?
I have a ticker plant which has 3 realtime subscribers of which one subscriber is slow.
q).z.W
7 | `long$()
8 | 969393 198 198 197 197 198 196 199 197 196 143 198 196 196 197 197 198 19..
9 | 199 198 198 143 197 199 197 197 197 197 199 196 199 145 196 198 198 198 1..
10| 198 196 198 144 199 198 198 198 196 197 196 199 198 143 199 198 197 198 1..
q)count each .z.W
7 | 0
8 | 85547
9 | 77931
10| 0
q)count each .z.W
7 | 0
8 | 191552
9 | 0
10| 0
Can the slow consumer get tickerplant killed or slow it down in production kdb+ systems receiving billions of records?
Yes, a slow consumer can kill a tickerplant. The slow consumer creates the output queue in the tickerplant and this output queue consumes memory. Eventually if it goes on long enough the tickerplant (or the machine it's running on) can run out of memory and abort.
Ideally a production tickerplant will have some form of monitoring which periodically keeps an eye on the output queues - if a queue gets beyond a certain threshold it should halt the subscription (temporarily remove the handle from the .u.w subscription dictionary, allow the queue to drain) and resume if/when the subscriber catches up. Or be more aggressive and close the subscribers connection entirely (hclose) which wipes the output queue.
If your system experiences a lot of queueing then the tickerplant might also need a daily garbage collect too (say at EOD) to make sure that output queues haven't caused it to hold unused memory (or maybe you want to keep it with unused memory so that it doesn't have to re-request memory from the OS next time there's a big queue)