I encounter a RuntimeError: Event loop is closed error when trying to run asyncio.run() multiple times with Firebase. Is this expected? Here is a simplified example of the issue:
import streamlit as st
import firebase_admin
from firebase_admin import credentials, firestore_async
import json
import asyncio
key_dict = json.loads(st.secrets["textkey"])
cred = credentials.Certificate(key_dict)
app = firebase_admin.initialize_app(cred)
db_async = firestore_async.client(app)
async def _upload_async():
dicts_to_upload = {
"doc_1": {"key_1": "val_1"},
"doc_2": {"key_2": "val_2"},
}
tasks = []
for key in dicts_to_upload.keys():
result_dict = db_async.collection("test-collection").document(key)
task = result_dict.set(dicts_to_upload[key])
tasks.append(task)
await asyncio.gather(*tasks)
asyncio.run(_upload_async())
print("\n\nSuccess1\n\n")
asyncio.run(_upload_async())
print("\n\nSuccess2\n\n")
When running this, the output is:
Success1
Traceback (most recent call last):
File "/Users/colinrichter/Documents/GitHub/YCN-Calculator/testing.py", line 100, in <module>
asyncio.run(_upload_async())
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/Users/colinrichter/Documents/GitHub/YCN-Calculator/testing.py", line 95, in _upload_async
await asyncio.gather(*tasks)
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/cloud/firestore_v1/async_document.py", line 131, in set
write_results = await batch.commit(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/cloud/firestore_v1/async_batch.py", line 60, in commit
commit_response = await self._client._firestore_api.commit(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/cloud/firestore_v1/services/firestore/async_client.py", line 1055, in commit
response = await rpc(
^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/retry/retry_unary_async.py", line 230, in retry_wrapped_func
return await retry_target(
^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/retry/retry_unary_async.py", line 160, in retry_target
_retry_error_helper(
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/retry/retry_base.py", line 212, in _retry_error_helper
raise final_exc from source_exc
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/retry/retry_unary_async.py", line 155, in retry_target
return await target()
^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/api_core/grpc_helpers_async.py", line 166, in error_remapped_callable
call = callable_(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/aio/_channel.py", line 150, in __call__
call = UnaryUnaryCall(
^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/grpc/aio/_call.py", line 565, in __init__
self._invocation_task = loop.create_task(self._invoke())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 434, in create_task
self._check_closed()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
sys:1: RuntimeWarning: coroutine 'UnaryUnaryCall._invoke' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
I also confirmed in Firebase that it is uploading the test-collection correctly the first time. However, when running it the second time I encounter the error. If I replace the firebase functions with a regular async function that does not use Firebase (like just adding something to doc_1 and doc_2) then it works as expected without this error.
I am doing this because I have a website on Streamlit where users fill out a form which updates some dictionaries, and then I upload those dictionaries to Firebase. I am trying to do this asynchronously to increase performance, but I can only get it to work the first time a user fills out the form. All subsequent times, I run into a RuntimeError: Event loop is closed error.
AmI doing something incorrectly, or is this a bug/feature of Firebase?
If you are happy with this design, just change the control flow lines at the end to use the same loop. Istead of:
asyncio.run(_upload_async())
print("\n\nSuccess1\n\n")
asyncio.run(_upload_async())
print("\n\nSuccess2\n\n")
Do
loop = asyncio.new_event_loop() # Maybe change this line to the very
# beggining, after the imports.
loop.run_until_complete(_upload_async())
print("\n\nSuccess1\n\n")
loop.run_until_complete(_upload_async())
print("\n\nSuccess2\n\n")
What is taking place there is that the firebase async client is binding itself to the running loop the first time it is actually used, and, when you call asyncio.run, a new loop is created. It probably would work if you'd recreate the app and db_async instances between calls.
The "right thing to do" however, would be to refactor this code and put all of these instances and calls out of "top level" code and into an asynchronous main function, though - the only line of code outside any function should be one calling the loop to run this controlling function.