Search code examples
pythonaiohttp

async for in python 3.6 to start jobs simultaneously


I have several http requests to fire simultaneously. I am trying to use async for to do this.

import asyncio

async def ticker(delay, to):
    for i in range(to):
        yield i
        print(i)
        await asyncio.sleep(delay) # instead of aiohttp request
        print(i, ' fin')

async def main():
    async for x in ticker(1,2):
        pass

if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

I keep getting subsequent calls with the following:

0
0 fin
1
1 fin

Instead I need the output as shown below:

0
1
0 fin
1 fin

Could you please advise me on how to do this?


Solution

  • The problem is that async for is exactly what you don't need.

    async for is designed to iterate while waiting for a task to complete between each iteration; you want to iterate (starting requests) without waiting for the previous task(s) to finish.

    You'll want something like

    async def do_request():
        await asyncio.sleep(1)
    
    async def main():
        await asyncio.gather(*[
            do_request() for i in range(10)
        ])
    

    Comment with a follow-up if that doesn't answer your question.