I'm trying request.get()
with proxies and I use try/except
.
If my proxy is wrong, I use except
, changing proxy and starting this function again.
After recursion I got all needed values and
TypeError: 'NoneType' object is not iterable
And I have no idea why?
P.S.: If the code works without calling Except, it works perfectly
def get_html_v2(url, proxy, userAgent):
data = None
status = None
userAgent = userAgent or ''
proxies = {
"http": "http://"+proxy,
"https":"http://"+proxy
}
try:
# Get request
response = requests.get(url=url, headers=userAgent, proxies=proxies)
# Get a status of the request
status = response.status_code
# Return request values and status
return response.text, int(status)
# If proxy doesn't work
except ConnectionError:
# Remove wrong proxy from List
proxiesList.remove(proxy)
# Get new proxy
proxy = random.choice(proxiesList)
# Start function again
get_html_v2(url, userAgent=userAgent, proxy=proxy)
It's unclear where the error is occurring, since the code above doesn't include any iteration. Would be better to include that section of the error.
Anyway, the error is probably occurring due to get_html_v2
in your except
block. You calling get_html_v2()
in the except block but not returning that result to the caller/previous loop of recursion, which you must do anyway, especially with recursion.
In the last line for your except
block, replace:
get_html_v2(url, userAgent=userAgent, proxy=proxy)
with:
return get_html_v2(url, userAgent=userAgent, proxy=proxy)
That way, you're effectively returning the value returned from the line return response.text, int(status)
in your try
block.
P.S.: If the code works without calling Except, it works perfectly
When the except
block gets called, since there's no return
, it implicitly returns None
. When it doesn't get called, it's returning response.text, int(status)