uWSGI thread has existing running event loop which causes Django's SynchronousOnlyOperation exception

Question

I have two Django views,

def view_that_accesses_orm(request): # say end point /useorm
  user = User.objects.first()
  ...

and

def view_that_creates_event_loop(request): # say endpoint /createloop
  client = AsycProvider()
  ... # do stuff with client

and AsyncProvider is something like

class AsyncProvider:
  def __init__(self):
    try:
      self.__loop = asyncio.get_event_loop()
    except RuntimeError as e:
      print(e) #no running event loop
      self.__loop = asyncio.new_event_loop()
      asyncio.set_event_loop(self.__loop)
    self.__session = aiohttp.ClientSession(loop=self.__loop)

  ... # other operations with asyncio.run_until_complete, asyncio.gather, and self.__session

Now the issue is that say if I have 1 process in uWSGI and 2 threads. Then they will be in a round robin fashion serving requests.

So scenario is:

1. User hits /createloop (given thread 1)
2. User hits /useorm (given thread 2)
3. User again hits /useorm (given thread 1)

Now in third scenario, sometimes the event loop is running and since Django 3.x detects a running event loop and disallows us to access ORM, I get the django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async. exception.

I am not sure how the event loop doesn't stop and persist in the thread.

Please explain what exactly could be the cause of this and what should be the fix?

Answer 1

The problem was that somewhere inside a method of AsyncProvider was using asyncio.set_event_loop(self.__loop).

So the same loop instance was being referenced in different threads (new loop wasn't being created). Now since this loop would sometimes also have some running logic and also if in a thread (which has referenced this loop) ORM was being accessed, Django threw the SynchronousOnlyOperation as it could detect running event loop.

Solved it by removing asyncio.set_event_loop(self.__loop) in the method.