Search code examples
python-3.xpython-asynciopython-datetime

datetime.strptime() is blocking asyncio.Queue.get()


In the following code datetime.strptime() is blocking asyncio.Queue.get() operation

import time
import asyncio
from datetime import datetime
from functools import partial

def write(queue):

    data = dict(fname = "Stack", lname = "Overflow")
    
    # If you comment the below line thing are working fine
    dob = datetime.strptime("2025-02-12", "%Y-%m-%d")

    queue.put_nowait(data)
    print("Writing to queue complete")
    time.sleep(3600)  # sleep forever

async def reader(queue):
    loop = asyncio.get_running_loop()
    print("This print means this thread is not blocked")
    while msg := await queue.get():
        print("Got my message", msg)
        loop.call_soon(print, msg)

async def main():
    queue = asyncio.Queue()
    loop = asyncio.get_running_loop()
    loop.run_in_executor(None, partial(write, queue))
    await reader(queue)

asyncio.run(main())

If I comment out datetime.strptime() function call I get the following output

Writing to queue complete
This print means this thread is not blocked
Got my message {'fname': 'Stack', 'lname': 'Overflow'}
{'fname': 'Stack', 'lname': 'Overflow'}

But if datetime.strptime() is not commented I get the following output

This print means this thread is not blocked
Writing to queue complete

Why is datetime.strptime() blocking asyncio.Queue.get() operation?


Solution

  • The problem isn't datetime.strptime. The problem is that it is not safe to call queue.put_nowait from outside the event loop. The docs specifically mention that

    asyncio queues are not thread-safe

    so you cannot safely call queue.put_nowait from an executor thread. The time needed to execute datetime.strptime simply changed the execution order enough for the unsafe call to cause a problem.