Let's say I am doing a larger data analysis in Jupyter/Ipython notebook with lots of time consuming computations done. Then, for some reason, I have to shut down the jupyter local server I, but I would like to return to doing the analysis later, without having to go through all the time-consuming computations again.
What I would like love to do is pickle or store the whole Jupyter session (all pandas dataframes, np.arrays, variables, ...) so I can safely shut down the server knowing I can return to my session in exactly the same state as before.
Is it even technically possible? Is there a built-in functionality I overlooked?
EDIT: based on this answer there is a %store magic which should be "lightweight pickle". However you have to store the variables manually like so:
#inside a ipython/nb session
foo = "A dummy string"
%store foo
closing seesion, restarting kernel
%store -r foo # r for refresh
print(foo) # "A dummy string"
which is fairly close to what I would want, but having to do it manually and being unable to distinguish between different sessions makes it less useful.
I think Dill (pip install dill) answers your question well.
Use dill.dump_session to save a Notebook session:
import dill
dill.dump_session('notebook_env.db')
Use dill.load_session to restore a Notebook session:
import dill
dill.load_session('notebook_env.db')
(source)