In the following code datetime.strptime() is blocking asyncio.Queue.get() operation
import time
import asyncio
from datetime import datetime
from functools import partial
def write(queue):
data = dict(fname = "Stack", lname = "Overflow")
# If you comment the below line thing are working fine
dob = datetime.strptime("2025-02-12", "%Y-%m-%d")
queue.put_nowait(data)
print("Writing to queue complete")
time.sleep(3600) # sleep forever
async def reader(queue):
loop = asyncio.get_running_loop()
print("This print means this thread is not blocked")
while msg := await queue.get():
print("Got my message", msg)
loop.call_soon(print, msg)
async def main():
queue = asyncio.Queue()
loop = asyncio.get_running_loop()
loop.run_in_executor(None, partial(write, queue))
await reader(queue)
asyncio.run(main())
If I comment out datetime.strptime() function call I get the following output
Writing to queue complete
This print means this thread is not blocked
Got my message {'fname': 'Stack', 'lname': 'Overflow'}
{'fname': 'Stack', 'lname': 'Overflow'}
But if datetime.strptime() is not commented I get the following output
This print means this thread is not blocked
Writing to queue complete
Why is datetime.strptime() blocking asyncio.Queue.get() operation?
The problem isn't datetime.strptime
. The problem is that it is not safe to call queue.put_nowait
from outside the event loop. The docs specifically mention that
asyncio queues are not thread-safe
so you cannot safely call queue.put_nowait
from an executor thread. The time needed to execute datetime.strptime
simply changed the execution order enough for the unsafe call to cause a problem.