Integrating Flask/Dill to dump/load server sessions

Question

I'm trying to integrate Flask with Dill to dump/load Python sessions on the server side. The code below has two functions, the first one sets the value of x to zero and imports the datetime library. The second one increments x by 1 and gets the timestamp.

The first function dumps the session and the second function loads it.

In the dump, the pickle file is generated correctly, but I cannot reuse x or get the timestamp.

This is the error when I try to execute x = x + 1 in the second function:

UnboundLocalError: local variable 'x' referenced before assignment

Can Dill be used with Flask? Do I need a different approach?

The code:

from flask import Flask
from dill import dump_session, load_session

app = Flask(__name__)
app.config['SECRET_KEY'] = 'super secret'

session_file = '/tmp/session.pkl'

@app.route('/start_counter')
def start_counter():
    import datetime
    x = 0
    dump_session(filename = session_file)
    return 'New counter started!'

@app.route('/count')
def count():
    load_session(filename = session_file)
    x = x + 1
    now = datetime.datetime.now()
    dump_session(filename = session_file)
    return str(x) + '-' + str(now)

Answer 1

How to fix?

To make things simple, you need a data structure to hold your application state. I would use a dict because it's simple, but you can define a class for that too.

The easy (and tedious) way is to call state = dill.load('filename')and dill.dump(object,'filename') every time you need your application state.

This will work if your application is small. If you need to maintain a proper application state, you should use a database.

Ok. But WHAT happened here?

There are no compatibility issues with dill and Flask.

When you call dill.dump_session() it saves the state of __main__.

But, when you increase x in function count(), it is undefined because it wasn't saved by dill.

An easy way to see that is to put a breakpoint() before x = x + 1 or print the content inside a try..except clause:

try:
   print(x)
except ee:
   print(ee)
   pass;
x = x + 1

So, it didn't worked because the variable x wasn't defined in __main__ but in in the scope of the start_counter() function and dill.load_session() restores the stuff in __main__.

And what does that __main__ stuff means?

Let's see that using a Repl:

~/$ python
Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

Perfect. We have an empty python interpreter. That dir() shows what we have in __main__.

Now we'll load some libraries, and assign a variable, and define a function just because we can:

>>> import pandas, numpy, dill, pickle, json, datetime
>>> foo = "bar"
>>> def functionWithUglyName():
...    print("yep")
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'datetime', 'dill', 'foo', 'functionWithUglyName', 'json', 'numpy', 'pandas', 'pickle']```

Well. That __main__ stuff looks more populated.

Now let's save the session and exit the Repl:

>>> dill.dump_session('session_01')
>>>  exit()

What happens when we load the session with `dill.load_session()' ?

Let's open another Repl to discover it:

>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

Ok. Just another empty python interpreter...

Let's load the session and see what happens:

>>> import dill
>>> dill.load_session('session_01')
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'datetime', 'dill', 'foo', 'functionWithUglyName', 'json', 'numpy', 'pandas', 'pickle']

It loaded the contents of __main__ as expected. Wait a second. It loaded the functionWithUglyName we defined before. Is it real?

>>> functionWithUglyName()
yep

Turns out that dill is really good at serializing stuff. Most of the time you'll just need to pickle some data, but dill can do much more... and it is great for debugging and testing.