Hi I want to use sqlalchemy to get connect to my table, get some column, do things with them and then move on. Problem is when i exit the function my ram space is not opened up. here is a dummy example.
from sqlalchemy import create_engine
import gc
mem = !free
dic = {}
dic_conn = {}
print(mem[1])
engine_ = create_engine(psql_string)
for tb in ['tb_1', 'tb_2', 'tb_3', 'tb_4',]:
conn = engine_.connect()
dic[tb] = conn.execute("select * from tb".format(tb))
dic_conn[tb] = conn
for k in dic_conn.keys(): dic_conn[k].close(); dic_conn[k] = None
for k in dic.keys(): dic[k].close(); dic[k] = None
engine_.dispose()
del dic_conn
del dic
del engine_
gc.collect()
mem = !free
print(mem[1])
i tried anything I could think of, let me know how I can free up that memory
sqlalchemy holds the buffer side on the client side, you can alternatively have the buffer on server side and that way sqlalchemy frees the buffer space. this command will do it.
conn = engine_.execution_options(stream_results=True).connect()