Search code examples
pythonasync-awaitaiohttp

multiple nonblocking tasks using asyncio and aiohttp


I am trying to perform several non blocking tasks with asyncio and aiohttp and I don't think the way I am doing it is efficient. I think it would be best to use await instead of yield. can anyone help?

def_init__(self):
    self.event_loop = asyncio.get_event_loop()

def run(self):
    tasks = [
        asyncio.ensure_future(self.subscribe()),
        asyncio.ensure_future(self.getServer()),]
    self.event_loop.run_until_complete(asyncio.gather(*tasks))
    try:
       self.event_loop.run_forever()

@asyncio.coroutine
def getServer(self):
    server = yield from self.event_loop.create_server(handler, ip, port)
    return server

@asyncio.coroutine
def sunbscribe(self):
    while True:
        yield from asyncio.sleep(10)
        self.sendNotification(self.sub.recieve())

def sendNotification(msg):
    # send message as a client

I have to listen to a server and subscribe to listen to broadcasts and depending on the broadcasted message POST to a different server.


Solution

  • According to the PEP 492:

    await , similarly to yield from , suspends execution of read_data coroutine until db.fetch awaitable completes and returns the result data.

    It uses the yield from implementation with an extra step of validating its argument. await only accepts an awaitable , which can be one of:

    So I don't see an efficiency problem in your code, as they use the same implementation.

    However, I do wonder why you return the server but never use it.

    The main design mistake I see in your code is that you use both:

    self.event_loop.run_until_complete(asyncio.gather(*tasks))
    try:
       self.event_loop.run_forever()
    

    From what I can see you just need the run_forever()

    Some extra tips:

    In my implementations using asyncio I usually make sure that the loop is closed in case of error, or this can cause a massive leak depending on your app type.

    try:
        loop.run_until_complete(asyncio.gather(*tasks))
    finally:  # close the loop no matter what or you leak FDs
        loop.close()
    

    I also use Uvloop instead of the builtin one, according to benchmarks it's much more efficient.

    import uvloop
    ...
        loop = uvloop.new_event_loop()
        asyncio.set_event_loop(loop)