Search code examples
pythonpython-requeststimeout

timeout not respected in request.get()


I can't understand why with this code the requests.get() returns immediately error and the 10 second timeout is not respected. If I remove the header parameter, so only requests.get(url, proxies={"http": proxy, "https": proxy}, verify=False,timeout=10), the timeout is respected and a request is returned successfully or failure if the timeout occurs. I need to insert headers and a timeout too to test proxies, how to do it?

from fake_useragent import  UserAgent
import requests
ua = UserAgent()

#get a list of proxy 

for i in range(1, len(proxies)):
    # Get a proxy from the pool
    proxy = next(proxy_pool)
    print("Request #%d" % i)
    try:
        response = requests.get(url, headers=ua.random(), proxies={"http": proxy, "https": proxy}, verify=False,timeout=10)
        break
    except:
        print("Skipping. Connnection error")

Solution

  • try with this

    ua = UserAgent()
    header = {'User-Agent':str(ua.random)}
    response = requests.get(url, headers=header, verify=False,timeout=10)