uwsgi worker hangs on exit, but only if it used async trio module

Question

I have a Django site that uses the trio_cdp package to generate PDFs using a headless Google Chrome. This package is async, but my Django project is sync, so it has to run inside trio.run()

It's also using uwsgi locks so that only one client can generate a PDF at a time (headless Chrome loads the page in a single virtual tab, so it can only do one at a time)

Here is the code:

import trio
import base64
import requests
from django.conf import settings
from trio_cdp import open_cdp, page, target

try:
    import uwsgi
    have_uwsgi = True
except ModuleNotFoundError:
    have_uwsgi = False

async def render_pdf_task(url, params):
    r = requests.get(url=settings.CDP_URL)
    if r.status_code == 200:
        out = r.json()
        ws_url = out[0]['webSocketDebuggerUrl']
    else:
        return None
    async with open_cdp(ws_url) as conn:
        targets = await target.get_targets()
        target_id = targets[0].target_id
        async with conn.open_session(target_id) as session:
            async with session.page_enable():
                async with session.wait_for(page.LoadEventFired):
                    await page.navigate(url)
                await trio.sleep(0.5)
                pdf = await page.print_to_pdf(**params)
                pdfdata = base64.b64decode(pdf[0])
        await conn.aclose()
    return pdfdata

def render_pdf(url, params):
    if have_uwsgi:
        uwsgi.lock(1)
    pdfdata = trio.run(render_pdf_task, url, params)
    if have_uwsgi:
        uwsgi.unlock(1)
    return pdfdata

Annoyingly, any uwsgi worker that has run this particular task will later hang on exit until it is forcibly killed. If uwsgi runs and nobody visits the PDF-generating page, all the uwsgi workers exit fine. And it is consistently the uwsgi workers that ran the render_pdf function that need to be killed.

For example, pid 20887 had run render_pdf, and later when trying to shut down uwsgi, this happened:

SIGINT/SIGQUIT received...killing workers...
worker 10 buried after 1 seconds
worker 9 buried after 1 seconds
worker 7 buried after 1 seconds
worker 6 buried after 1 seconds
worker 5 buried after 1 seconds
worker 4 buried after 1 seconds
worker 3 buried after 1 seconds
worker 2 buried after 1 seconds
worker 1 buried after 1 seconds
Tue Jan 25 22:44:42 2022 - worker 8 (pid: 20887) is taking too much time to die...NO MERCY !!!
worker 8 buried after 1 seconds
goodbye to uWSGI.

How can I fix this? Any help is much appreciated :)

Answer 1

I was able to solve this myself. uWSGI's handler for SIGINT is overridden by trio.run(), but only if trio.run() is in the main thread. I solved this by running it in another thread.