Search code examples
pythonwindowspython-requestsputtysocks

How do I use python Requests with a Putty SOCKS Proxy on Windows?

I’ve setup Putty to create a Socks proxy to a server. I’m able to use Chrome and Firefox to get to the host. When I use requests I get an error that seems to indicate something wrong with the connection to Putty. I’ve tried to set the HTTP_PROXY env variable and I’ve tried to pass the proxies argument. The env variable method produces a “failed to parse” error and the proxies method produces the max retries exceeded.

Setting the env produces this error when calling requests.get("http://10.122.116.17:8080/")

requests.packages.urllib3.exceptions.LocationParseError: Failed to parse: Failed to parse: "localhost:1088"

Passing the proxies argument produces this error when calling requests.get("http://10.122.116.17:8080/",proxies=proxies)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=1088): Max retries exceeded with url: http://10.122.116.17:8080/ (Caused by <class 'httplib.BadStatusLine'>: '')

I’ve installed Wireshark and am filtering on my proxy server which is at 216.255.67.119. When I run Firefox I see traffic to this address. When I run Python Requests I don’t see any traffic to the server. So I suspect that the proxy mechanism in requests is not getting to my Putty session.

Does anyone have any hints or suggestions what might be happening here? I’ve googled until my fingers hurt!

Thanks in advance!!!

Here’s what the server looks like when I use Firefox…

Tomcat on Private Server

---------------- cmd output -------------------------

C:\Users\patman>set HTTP_PROXY="localhost:1088"
C:\Users\patman>echo %HTTP_PROXY%
"localhost:1088"
C:\Users\patman>set HTTPS_PROXY="localhost:1088"
C:\Users\patman>python
Enthought Python Distribution -- www.enthought.com
Version: 7.3-2 (64-bit)

Python 2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 15:20:16) [MSC v.1500 64 bit (AMD64)] on win32
Type "credits", "demo" or "enthought" for more information.
>>> import requests
>>> requests.get("http://10.122.116.17:8080/")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\api.py",
line 55, in get
    return request('get', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\api.py",
line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\sessions
.py", line 346, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\sessions
.py", line 449, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\adapters
.py", line 263, in send
    conn = self.get_connection(request.url, proxies)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\adapters
.py", line 188, in get_connection
    conn = ProxyManager(self.poolmanager.connection_from_url(proxy))
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\packages
\urllib3\poolmanager.py", line 122, in connection_from_url
    u = parse_url(url)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\packages
\urllib3\util.py", line 156, in parse_url
    raise LocationParseError("Failed to parse: %s" % url)
requests.packages.urllib3.exceptions.LocationParseError: Failed to parse: Failed to parse: "localhost:1088"

>>> >>> proxies = {
...           "http": "localhost:1088",
...           "https": "localhost:1088",
...         }
>>> requests.get("http://10.122.116.17:8080/",proxies=proxies)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\api.py",
line 55, in get
    return request('get', url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\api.py",
line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\sessions
.py", line 346, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\sessions
.py", line 449, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python27\lib\site-packages\requests-1.2.0-py2.7.egg\requests\adapters
.py", line 318, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=1
088): Max retries exceeded with url: http://10.122.116.17:8080/ (Caused by <clas
s 'httplib.BadStatusLine'>: '')
>>> requests.__version__
'1.2.0'
>>> import urllib3
>>> urllib3.__version__
'1.7.1'
>>>

Putty settings… enter image description here

Solution

  • Requests does not support SOCKS proxies at this time. We're hoping to get support in the underlying urllib3 library, but there's no explicit time-frame on that. It's tracked by this issue.

    Edit in 2016: As of Requests 2.10, Requests now has support for SOCKS proxies. You can get the support by installing the socks extra: pip install requests[socks].