I have an endpoint on my appengine app (/start) that I use to kick off a process. In the python 2 environment I used deferred to run the process outside of the request context. On appengine flex, you can run background threads that live outside of the request environment. At first look, I thought this was also true of the python 3 standard environment, however at a closer look it contains this note:
However note that new threads or processes may not run after the inbound request is served.
Does this mean that once the request is served the background thread that I started will be killed? If so, what is the best way to start background work?
I could re-create the deferred library by creating a queue in cloud task and pickling everything, but that's a last resort.
From the Task Queue section of the Understanding differences between Python 2 and Python 3 on the App Engine standard environment guide:
You can use Cloud Tasks (beta) to access the Task Queue service.
In cases where pull queues are appropriate, such as queuing up tasks or messages that will be pulled and processed by separate workers, Cloud Pub/Sub can be a good alternative. Cloud Pub/Sub offers similar functionality and delivery guarantees.
So yes, cloud tasks is the recommended solution.
But you don't really need to re-create the deferred library - that one was built on top of the push task queues. The only advantage of the deferred library over the push tasks was that you didn't have to pre-register handlers for the tasks (you'd just pass the function to be executed and its args). But that's not true for the cloud tasks - they do require GAE handlers.