I am working on migrating an existing python GAE (Google App Engine) standard environment app to the flexible environment. I read through the guide and decided to try out the python-compact runtime, as it's always good to re-use as much code as possible.
In the standard environment app, we use background_thread.start_new_background_thread() to spawn a bunch of infinite-loop threads to work on some background work forever. However, I couldn't get start_new_background_thread working in the flexible environment, even for some really simple app. Like this sample app: github.com/GoogleCloudPlatform/python-docs-samples/tree/master/appengine/background
I keep getting the following error while running the app in the cloud (it works fine locally though).
I debugged into it by using the cloud debugger, but there was no any error message available at all while the exception was raised in the background_thread.py
Any idea how I can run a long-live background thread in the flexible environment with python-compact runtime? Thanks!
One of the differences between App Engine standard and App Engine flexible is that with Flex we're really just running a docker container. I can think of 2 approaches to try out.
App Engine standard enforces a sandbox that mostly means no direct use of threads or processes. With Flex, you should be able to just use the standard Python lib for starting a new sub process:
https://docs.python.org/3/library/subprocess.html
If that doesn't work, another approach you could take here is to customize the docker image you're using in Flex, and use supervisord to start multiple processes. First, generate the dockerfile by cd-ing into folder with your sources and running:
gcloud preview app gen-config --custom
This will create a Dockerfile that you can customize. Now, you are going to want to start 2 processes - the process we were starting (I think for python-compat it's gunicorn) and your background process. The easiest way to do that with docker is to use supervisord:
https://docs.docker.com/engine/admin/using_supervisord/
After modifying your Dockerfile and adding a supervisord.conf, you can just deploy your app as you normally would with gcloud preview app deploy.
Hope this helps!
