KDB/Q memory consumption

Question

I have a KDB/Q databse which has around ~2M records per day consuming about ~2G of memory. At end of day it runs some reporting stuff doing joins between the tables and outputting result into files on disk. During the computation the memory usage grows to ~15G. My problem is that once this operation finishes the memory is never released back and until the DB is restarted it consumes all the 15G of memory.

I would like to tell KDB to unload some tables from memory (not drop them though) but I don't want to restart the DB since some other apps are still connecting to it.

Is there a way to tell KDB to unload something from memory?

EDIT:

If anyone finds it interesting I suggest to have a look on .Q.gc[] for KDB 2.5+, looks promising.

Answer 1

Here is the sum up of my research:

• KDB prior to ver. 2.5 allocates 64MB memory chunks on need and never release them. It is able to reuse them though.
• recent KDB versions allow .Q.gc[] call which is on-request call to garbage collector (KDB uses ref. counting btw.)
• this is especially useful when you invoke some memory intensive computation which allocates lot of memory (in my case it was ~20gB) and you want to release the memory after the computation finishes.
• you can always consider putting the memory intensive script into a separate Q process so the memory will be released once the script finishes