How to write to the Flask Session from a child process

I have a flask web app that does image processing. The parent application calls a function that creates several temporary images, writes them to disk, and stores their paths in the flask Session. That workflow is leaking memory -- the RAM allocated to the function call is not released until the webserver is restarted.

As a workaround, I am trying to call the function from a separate process, but encounter the following problems:

Keys/Values added to the Session within the function are not propagated to the parent namespace
Values written to multiprocessing.Value objects within the function are not propagated to the parent namespace

To demonstrate the problems, I've created the 3 following minimal examples.

app.py

from flask import Flask, session
from flask_session import Session
from multiprocessing import Process, Value

app = Flask(__name__.split('.')[0])
app.secret_key = "secret_key"

SESSION_TYPE = 'filesystem'
app.config.from_object(__name__)
Session(app)


def write_to_session(session):
    session['inside_function'] = "inside"
    print(session.items())

# This route demonstrates the desired behavior: the session is modified by the function.
@app.route('/baseline', methods=['GET'])
def baseline():
    session.clear()
    session['outside_function'] = "outside"

    text = ""
    text += "<p>Initial session data <br />" + str(session.items()) + "</p>"
    write_to_session(session)
    text += "<p>Session after function call <br />" + str(session.items()) + "</p>"

    return text

# When calling the function from a process, the session is correctly
# modified within the function, but the changes are not propagated back
# to the parent scope.
@app.route('/multi', methods=['GET'])
def multi():
    session.clear()
    session['outside_function'] = "outside"

    text = ""
    text += "<p>Initial session data <br />" + str(session.items()) + "</p>"

    process = Process(target=write_to_session, args=(session,))
    process.start()
    process.join() # waits for process to end

    text += "<p>Session after function call <br />" + str(session.items()) + "</p>"

    return text

# This route attempts to return a function value from the process using a 
# multiprocessing.Value object. The value assigned within the function is 
# replaced by '\x01' before being propagated back to the parent scope.
@app.route('/value', methods=['GET'])
def value():
    from ctypes import c_wchar_p

    def assign_string(cstring):
        cstring.value = "inside"

    cstring = Value(c_wchar_p, "Hello initial!")
    text = "<p>Initial Value:<br />" + str(cstring.value) + "</p>"

    process = Process(target=assign_string, args=(cstring,))
    process.start()
    process.join() 

    text += "<p>Value after function call:<br />" + str(cstring.value) + "</p>"

    return text


if __name__ == '__main__':
    app.run(host='localhost', port=5000, debug=True, use_reloader=True)

Results

The expected behavior. The Session is modified by the function.

http://localhost:5000/baseline

Initial session data
dict_items([('outside_function', 'outside')])
Session after function call
dict_items([('outside_function', 'outside'), ('inside_function', 'inside')])

Writing to the Session from the child process: the key + value inserted by the function are not propagated to parent scope.

http://localhost:5000/multi

Initial session data
dict_items([('outside_function', 'outside')])
Session after function call
dict_items([('outside_function', 'outside')])

Passing a string via a multiprocessing.Value object: the assigned string is replaced by a boolean true: '\x01'

http://localhost:5000/value

Initial Value:
Hello initial!
Value after function call:

Anaconda Environment

name: flask
channels:
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_gnu
  - bzip2=1.0.8=h4bc722e_7
  - ca-certificates=2024.8.30=hbcca054_0
  - cachelib=0.13.0=pyhd8ed1ab_0
  - click=8.1.7=unix_pyh707e725_0
  - flask=2.2.0=pyhd8ed1ab_0
  - flask-session=0.5.0=pyhd8ed1ab_0
  - importlib-metadata=8.4.0=pyha770c72_0
  - itsdangerous=2.2.0=pyhd8ed1ab_0
  - jinja2=3.1.4=pyhd8ed1ab_0
  - ld_impl_linux-64=2.40=hf3520f5_7
  - libexpat=2.6.3=h5888daf_0
  - libffi=3.4.2=h7f98852_5
  - libgcc=14.1.0=h77fa898_1
  - libgcc-ng=14.1.0=h69a702a_1
  - libgomp=14.1.0=h77fa898_1
  - libnsl=2.0.1=hd590300_0
  - libsqlite=3.46.1=hadc24fc_0
  - libuuid=2.38.1=h0b41bf4_0
  - libxcrypt=4.4.36=hd590300_1
  - libzlib=1.3.1=h4ab18f5_1
  - markupsafe=2.1.5=py311h9ecbd09_1
  - ncurses=6.5=he02047a_1
  - openssl=3.3.2=hb9d3cd8_0
  - pip=24.2=pyh8b19718_1
  - python=3.11.0=he550d4f_1_cpython
  - python_abi=3.11=5_cp311
  - readline=8.2=h8228510_1
  - setuptools=73.0.1=pyhd8ed1ab_0
  - tk=8.6.13=noxft_h4845f30_101
  - tzdata=2024a=h8827d51_1
  - werkzeug=2.2.2=pyhd8ed1ab_0
  - wheel=0.44.0=pyhd8ed1ab_0
  - xz=5.2.6=h166bdaf_0
  - zipp=3.20.1=pyhd8ed1ab_0
prefix: /home/rh/.local/share/mambaforge-pypy3/envs/flask

What am I missing here? What is the proper way to

Write to the Session from a separate process?
Return a string from a separate process?

Solution

You are not missing anything. Child process doesn't know anything about flask app, app context and request context. From docs:

The context is unique to each thread (or other worker type). request cannot be passed to another thread, the other thread has a different context space and will not know about the request the parent thread was pointing to

What is the proper way?

Just use Redis and implement everything you wish