Does gevent monkey patched Queue.put yield (context switch)?

Question

import gevent.monkey
gevent.monkey.patch_all()
import gevent
from queue import Queue
import random
import time


def getter(q):
    while True:
        print('getting')
        v = q.get()
        print(f'got {v}')

def putter(q):
    while True:
        print(f'start putting')
        v = int(random.random() * 1000)
        # `put_nowait` also seems to yield
        # q.put(v)
        q.put_nowait(v)
        print(f'done putting with {v}')

        if random.random() < 0.5:
            print(f'yield')
            time.sleep(0)


queue = Queue()


gevent.spawn(getter, queue)
gevent.spawn(putter, queue)

time.sleep(1000)

Doesn't matter if I use queue.put or queue.put_nowait, I saw logs like

# start putting
# got 25
# getting
# done putting with 535

Does that suggest gevent might do context switch each time it executes queue.put?

Update

I modified the code a bit

flag = True

def getter(q):
    while True:
        print('getting')
        global flag
        flag = False
        v = q.get()
        print(f'got {v}')

def putter(q):
    v = 0
    while True:
        print(f'start putting {v}')
        global flag
        flag = True
        # `put_nowait` also seems to yield
        # q.put(v)
        q.put_nowait(v)
        if not flag:
            raise Exception('yield happened')

        print(f'done putting with {v}')
        v += 1

        time.sleep(0)
        # If I sleep with non-zero, the above seems to only yield once at the beginning.
        # time.sleep(0.000001)


queue = Queue()


def myTracer(event, args):
    src, target = args
    if event == "switch":
        # print("from %s switch to %s" % (src, target))
        # Print to stdout like the rest of the code. Otherwise the order of stdout & stderr is not guaranteed.
        traceback.print_stack(file=sys.stdout)
    elif event == "throw":
        print("from %s throw exception to %s" % (src, target))


# greenlet.settrace(myTracer)

gevent.spawn(getter, queue)
putter(queue)

With patched Python Queue,

• If I sleep(0), it might do context switch.
• If I sleep(0.0000001), it only yield once at the beginning.
• If I don't sleep at all, it only yield once at the beginning, and the getter doesn't have a chance to run afterwards.

Looking at the stack trace, I found Queue.put calls notify, which calls lock.acquire(0). It was then patched so that it calls sleep() in gevent/thread.py

If I use gevent.queue.Queue instead of Python Queue, it doesn't seem to do context switch.

Answer 1

You can use greenlet.settrace with a callback function to detect context switches.
Adding that to your code shows that both put and put_nowait do context switches.

...

def myTracer(event, args):
    src, target = args
    if event == "switch":
        print("from %s switch to %s" % (src, target))
    elif event == "throw":
        print("from %s throw exception to %s" % (src, target))


greenlet.settrace(myTracer)


queue = Queue()
gevent.spawn(getter, queue)
gevent.spawn(putter, queue)

time.sleep(5)

You gonna see a lot of 'switching' debug messages in stdout:

...
done putting with 106
start putting
from <Greenlet at 0x10418e150: putter(<queue.Queue object at 0x1034d9460>)> switch to <Hub '' at 0x10416a400 select default pending=0 ref=3 thread_ident=0x106d15dc0>
from <Hub '' at 0x10416a400 select default pending=0 ref=1 thread_ident=0x106d15dc0> switch to <Greenlet at 0x10418e150: putter(<queue.Queue object at 0x1034d9460>)>
done putting with 487
start putting
....

Edit/Note:

I am using Python3.8 and gevent==20.9.0. When testing I removed the whole if-condition if random.random() ..., but piping stdout to a file and searching for getter context switches and getting both are present in stdout.

I didn't investigate further, but if you have a look at the source code you can see there is an explicit call getter.switch(getter) probably that's what's causing the context switches.