Search code examples
pythonpython-requeststypeerrornonetype

After recursion I got an Error (TypeError: 'NoneType' object is not iterable)


I'm trying request.get() with proxies and I use try/except. If my proxy is wrong, I use except, changing proxy and starting this function again.

After recursion I got all needed values and

TypeError: 'NoneType' object is not iterable

And I have no idea why?

P.S.: If the code works without calling Except, it works perfectly

def get_html_v2(url, proxy, userAgent):
    data = None
    status = None
    userAgent = userAgent or ''
    proxies = {
        "http": "http://"+proxy,
        "https":"http://"+proxy
    }
    try:
        # Get request
        response = requests.get(url=url, headers=userAgent, proxies=proxies)
        # Get a status of the request
        status = response.status_code
        # Return request values and status
        return response.text, int(status)
    # If proxy doesn't work
    except ConnectionError:
        # Remove wrong proxy from List
        proxiesList.remove(proxy)
        # Get new proxy
        proxy = random.choice(proxiesList)
        # Start function again
        get_html_v2(url, userAgent=userAgent, proxy=proxy)

Solution

  • It's unclear where the error is occurring, since the code above doesn't include any iteration. Would be better to include that section of the error.

    Anyway, the error is probably occurring due to get_html_v2 in your except block. You calling get_html_v2() in the except block but not returning that result to the caller/previous loop of recursion, which you must do anyway, especially with recursion.

    In the last line for your except block, replace:

     get_html_v2(url, userAgent=userAgent, proxy=proxy)
    

    with:

     return get_html_v2(url, userAgent=userAgent, proxy=proxy)
    

    That way, you're effectively returning the value returned from the line return response.text, int(status) in your try block.

    P.S.: If the code works without calling Except, it works perfectly

    When the except block gets called, since there's no return, it implicitly returns None. When it doesn't get called, it's returning response.text, int(status)