Look, people. We have a question about gevent.pool class and pool.wait_available() method, both code snippets
1.
def fetch(url):
print 'start fetching...', url
data = urllib2.urlopen(url)
print url,':',data.code
urls = ['http://www.google.ru', 'http://www.s-str.ru', 'http://www.vk.com', 'http://www.yandex.ru', 'http://www.xxx.com']
pool = Pool(2)
def producer():
for url in urls:
pool.spawn(fetch, url)
pool.join()
p = gevent.spawn(producer)
p.join()
2.
def fetch(url):
print 'start fetching...', url
data = urllib2.urlopen(url)
print url,':',data.code
urls = ['http://www.google.ru', 'http://www.s-str.ru', 'http://www.vk.com', 'http://www.yandex.ru', 'http://www.xxx.com']
pool = Pool(2)
def producer():
for url in urls:
pool.wait_available()
pool.spawn(fetch, url)
pool.join()
p = gevent.spawn(producer)
p.join()
give us similar results:
start fetching... http://www.google.ru
start fetching... http://www.s-str.ru
http://www.google.ru : 200
start fetching... http://www.vk.com
http://www.s-str.ru : 200
start fetching... http://www.yandex.ru
http://www.yandex.ru : 200
start fetching... http://www.xxx.com
http://www.vk.com : 200
http://www.xxx.com : 200
Can anyone explain the meaning of wait_available() method? And possible cases of it's usage.
=======update======== I already monkey pathched it, it works correctly, all I want to know - is the difference between theese two code snippets.
Working with gevent you need to patch standard module before.
>>> import gevent.monkey
>>> gevent.monkey.patch_all()
>>> ...
>>> p = gevent.spawn(producer)
>>> p.join()
start fetching... http://www.google.ru
start fetching... http://www.s-str.ru
http://www.google.ru : 200
start fetching... http://www.vk.com
http://www.vk.com : 200
start fetching... http://www.yandex.ru
http://www.yandex.ru : 200
start fetching... http://www.xxx.com
http://www.xxx.com : 200
http://www.s-str.ru : 200
You can see, that pool.wait_available() works predictable.
Update
Pool works the same way only for spawn function (it will wait for available "slot" in pool). If you need to provide other functionality based on Pool state (logging, tracing, monitoring) - you definitely will use functions like wait_available, free_count etc. If you only need to spawn new green thread - you can rely on Pool implementation.