Search code examples
pythonmultithreadingeventlet

What are the benefits of using the Eventlet module in python over the threading module?

Specifically the GreenPool class in Eventlet. I have tested some code to upload large files to S3 as individual pieces of a multipart upload. What I have noticed so far is that when using eventlet the CPU usage is much lower. Just looking for other pros and cons for Eventlet over just using threading. Thanks.

Solution

  • Basically, Eventlet green threads are to be considered a lightweight analog of OS threads for all practical purposes. Pros:

    • cheaper to create in terms of CPU, memory and syscalls (0)
    • cheaper to switch; this is especially true in Python 2.x where each thread actively tries to grab GIL which wastes CPU.

    Cons:

    • important since many green threads operate within one OS thread, when a syscall (e.g. open(2)) in one of them blocks OS thread, all of the green threads are blocked too.
    • no SMP (multicpu/multicore); but then with GIL, this is also true for OS threads in Python. With greenlet[1] this restriction is more strict since it's not possible for some C extension to release GIL to allow other green threads to proceed.

    You may also find this answer useful: Is a greenthread equal to a "real" thread

    [1] "threading" library used by Eventlet https://github.com/python-greenlet/greenlet