Search code examples
pythonpython-3.xmultithreadingflaskpython-multithreading

Threading an external script within a Flask app


I have a Flask app that is using external scripts to perform certain actions. In one of the scripts, I am using threading to run the threads.

I am using the following code for the actual threading:

for a_device in get_devices:
    my_thread = threading.Thread(target=DMCA.do_connect, args=(self, a_device, cmd))
    my_thread.start()

main_thread = threading.currentThread()
for some_thread in threading.enumerate():
    if some_thread != main_thread:
        some_thread.join()

However, when this script gets ran (from a form), the process will hang and I will get a continuous loading cycle on the webpage.
Is there another way to use multithreading within the app?


Solution

  • Implementing threading by myself in a Flask app has always ended in some kind of disaster for me. You might want to use a distributed task queue such as Celery. Even though it might be tempting to spin off threads by yourself to get it finished faster, you will start to face all kinds of problems along the way and just end up wasting a lot of time (IMHO).

    Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

    The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

    Here are some good resources that you can use to get started