Python: Concurrent.Futures Error [TypeError: 'NoneType' object is not callable]

Question

So I managed to get asyncio / the Google CSE API to work together.... When I run my code on PyCharm, I am able to print my results out. However, at the very end of the printed stuff is the error "TypeError: 'NoneType' object is not callable".

I suspect it has something to do with my list and maybe the loop trying to search for another term even though I'm at the end of the list...

Also.. this is my first question post so feel free to offer suggestions on how to ask questions better as well

Thoughts?

searchterms = ['cheese',
    'hippos',
    'whales',
    'beluga']

async def sendQueries(queries, deposit=list()):
    with concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:
        loop = asyncio.get_event_loop()
        futures = [
            loop.run_in_executor(
                executor,
                searching(queries)
            )
        ]
        for response in await asyncio.gather(*futures):
            deposit.append(response.json())
        return deposit

def running():
     loop = asyncio.get_event_loop()
     loop.run_until_complete(loop.create_task(sendQueries(searchterms)))
     loop.close()

print(running())
print(str(time.time() - x))

My error can be traced back to "for response in await asyncio.gather(*futures):"

For your reference, searching(queries) is simply the function for my Google CSE API call.

Answer 1

The problem is in the call to run_in_executor:

    futures = [
        loop.run_in_executor(
            executor,
            searching(queries)
        )
    ]

run_in_executor accepts a function to execute. The code doesn't pass it a function, it calls a function, searching, and passes run_in_executor the return value of that call. That has two consequences:

1. the code doesn't work as intended - it invokes the searches one by after the other, not in parallel;

2. it displays an error complaining of an attempt by run_in_executor to invoke the None return value of searching(...). Confusingly, the error is raised only later, when awaiting the futures returned by run_in_executor, by which time all the searches have actually finished.

The correct way to call run_in_executor would be something like:

    futures = [
        loop.run_in_executor(executor, searching, queries)
    ]

Note how the searching function is now only mentioned rather than used.

Also, if you are only using asyncio to invoke synchronous calls in run_in_executor, you are not really benefiting from its use. You could get the same effect using threading-based tools from concurrent.futures directly, but without the need to adjust your whole program to asyncio. run_in_executor is meant to be used sparingly, either for occasional interfacing with legacy APIs that don't provide an async front-end, or for CPU-heavy code which cannot be meaningfully converted to coroutines.