Hello, currently I am working on a application that sends data to the frontend in Realtime through using django-channels. I have been able to do this for all models that are created when they page is opened, however I have not yet been able to grab previous models for when the page has not been opened yet.
Below was one of my first attempts at getting this system working, this would be called when the consumer connect method would be fired, and after the connection was accepted I would run this for loop
for app in Application.objects.all():
app_json = ApplicationSerializer(app).data
await self.send_json({'action': 'create', 'data': app_json})
When running this code I would get the error message of
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
This makes sense since the consumers connect method is asynchronous, so following the advice of the message I went ahead and used sync_to_async
for app in sync_to_async(Application.objects.all)():
app_json = ApplicationSerializer(app).data
await self.send_json({'action': 'create', 'data': app_json})
I decided to use sync_to_async around the Application objects since the error message highlighted the for loop line itself and also I know that the applicationSerializer would be working correctly since I have used it in other async methods just fine
This results in TypeError: 'coroutine' object is not iterable
I am new to both django-channels and asynchronous, if I knew that channels was going to be heavily based on asynchronous I would have studied the system before I started this project.
I have looked on other stack overflow posts and saw that potentially mapping the data might be a solution but I am not 100% sure if this is a asynchronous problem or with django-channels
What are some other things I can try?
Based off of Kens suggestion I went ahead and tried the following code:
apps = await database_sync_to_async(Application.objects.all)()
for app in apps:
app_json = ApplicationSerializer(app).data
print(app_json)
This returned the stacktrace
Exception inside application: You cannot call this from an async context - use a thread or sync_to_async.
Traceback (most recent call last):
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\staticfiles.py", line 44, in __call__
return await self.application(scope, receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\routing.py", line 71, in __call__
return await application(scope, receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\sessions.py", line 47, in __call__
return await self.inner(dict(scope, cookies=cookies), receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\sessions.py", line 254, in __call__
return await self.inner(wrapper.scope, receive, wrapper.send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\auth.py", line 181, in __call__
return await super().__call__(scope, receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\middleware.py", line 26, in __call__
return await self.inner(scope, receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\routing.py", line 150, in __call__
return await application(
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\consumer.py", line 94, in app
return await consumer(scope, receive, send)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\consumer.py", line 58, in __call__
await await_many_dispatch(
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\utils.py", line 51, in await_many_dispatch
await dispatch(result)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\consumer.py", line 73, in dispatch
await handler(message)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\generic\websocket.py", line 196, in websocket_receive
await self.receive(text_data=message["text"])
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\channels\generic\websocket.py", line 259, in receive
await self.receive_json(await self.decode_json(text_data), **kwargs)
File "D:\HomeAutomation\HomeAutomation\consumers.py", line 28, in receive_json
for app in apps:
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\django\db\models\query.py", line 287, in __iter__
self._fetch_all()
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\django\db\models\query.py", line 1308, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\django\db\models\query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\django\db\models\sql\compiler.py", line 1154, in execute_sql
cursor = self.connection.cursor()
File "C:\Users\Devin\.virtualenvs\HomeAutomation-l9uhKhE0\lib\site-packages\django\utils\asyncio.py", line 24, in inner
raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
This code sync_to_async(Application.objects.all)() creates a coroutine which needs to be awaited and not used directly. For your specific case, it would be better to first get the data outside the for loop and then proceed. Also, you may want to use database_sync_to_async instead.
from channels.db import database_sync_to_async
apps = await database_sync_to_async(Application.objects.all)()
for app in apps:
app_json = ApplicationSerializer(app).data
await self.send_json({'action': 'create', 'data': app_json})
As a side note, ensure that there will be no database queries when the serializer serialized the app instance. To do so, use select_related and prefetch_related to prefetch any nested objects that may be needed
UPDATE: Yes, due to the lazy nature of some Django ORM methods like .all(), the database won't actually be hit at the point of calling the method but at the point when it is being used. A common way to force it to execute immediately, is to convert the queryset to a list.
Try this:
from channels.db import database_sync_to_async
apps = await database_sync_to_async(list)(Application.objects.all())
for app in apps:
app_json = ApplicationSerializer(app).data
await self.send_json({'action': 'create', 'data': app_json})