Search code examples
pythonsqlalchemygarbage-collection

how can i remove sqlalchemy from ram after exiting function


Hi I want to use sqlalchemy to get connect to my table, get some column, do things with them and then move on. Problem is when i exit the function my ram space is not opened up. here is a dummy example.

from sqlalchemy import create_engine
import gc
mem = !free
dic = {}
dic_conn = {}
print(mem[1])
engine_ = create_engine(psql_string)
for tb in ['tb_1', 'tb_2', 'tb_3', 'tb_4',]:
    conn = engine_.connect()
    dic[tb] = conn.execute("select * from tb".format(tb))
    dic_conn[tb] = conn
for k in dic_conn.keys(): dic_conn[k].close(); dic_conn[k] = None
for k in dic.keys(): dic[k].close(); dic[k] = None
engine_.dispose()
del dic_conn
del dic
del engine_
gc.collect()
mem = !free
print(mem[1])

i tried anything I could think of, let me know how I can free up that memory


Solution

  • sqlalchemy holds the buffer side on the client side, you can alternatively have the buffer on server side and that way sqlalchemy frees the buffer space. this command will do it.

    conn = engine_.execution_options(stream_results=True).connect()