Search code examples
pythoneventlet

python eventlet - file async(non-blocking) io

According to eventlet document :

It uses epoll or libevent for highly scalable non-blocking I/O.

I know non-blocking I/O here covers the case when you interact with sockets. After the socket gets greened/patched by eventlet, the socket connection becomes non-blocking.

My questions is: Does this non-blocking I/O here covers file I/O as well?

Solution

  • No. That would either

    • True async IO with callbacks: impose unexpected restrictions; good on Windows and FreeBSD though
    • Non-blocking IO: disable filesystem cache (at least on Linux and FreeBSD) and still not guarantee that file IO would not block (at least on Linux on commonly used file systems)
    • Thread pool: incur unexpected performance and memory costs

    Thread pool is the easiest, most portable and most predictable way though. You could easily wrap your calls with eventlet.tpool.execute.