python: Task got bad yield:

Question

I am aiming to make parallel requests to the list of endpoints, hence using asyncio ensure_future can somone please take a look and give me an idea on how to fix errors (python3.6.7)

import asyncio
import treq


async def main_aysnc():
    loop = asyncio.get_event_loop()
    await start_coros_parllel()

async def start_coros_parllel():
    config = {}
    config['services'] = [ 
        b'https://www.google.com',
        b'https://www.yahoo.com',
        b'https://www.facebook.com'
        ]
    results = await asyncio.gather(*[asyncio.ensure_future(treq.get(service)) for service in config['services']])
    

if __name__ == "__main__":
    asyncio.get_event_loop().run_until_complete(main_aysnc())

LOGS

Traceback (most recent call last):
  File "2cr.py", line 35, in <module>
    asyncio.get_event_loop().run_until_complete(main_aysnc())
  File "/Users/vchauhan/.pyenv/versions/3.6.7/lib/python3.6/asyncio/base_events.py", line 473, in run_until_complete
    return future.result()
  File "2cr.py", line 7, in main_aysnc
    await start_coros_parllel()
  File "2cr.py", line 20, in start_coros_parllel
    results = await asyncio.gather(*[asyncio.ensure_future(treq.get(service)) for service in config['services']])
  File "/Users/vchauhan/.pyenv/versions/3.6.7/lib/python3.6/asyncio/tasks.py", line 537, in _wrap_awaitable
    return (yield from awaitable.__await__())
RuntimeError: Task got bad yield: <Deferred at 0x104cc5048>

Answer 1

The problem is that you should not use asyncio with treq.

According to documentation:

treq depends on a recent Twisted and functions on Python 2.7 and Python 3.3+ (including PyPy).

If you want to use asyncio you have to use some other http client framework e.g. aiohttp.

If you need any example on how to use aiohttp client, feel free to ask.