Search code examples
pythonurllib2socks

Using urllib2 with SOCKS proxy


Is it possible to fetch pages with urllib2 through a SOCKS proxy on a one socks server per opener basic? I've seen the solution using setdefaultproxy method, but I need to have different socks in different openers.

So there is SocksiPy library, which works great, but it has to be used this way:

import socks
import socket
socket.socket = socks.socksocket
import urllib2
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "x.x.x.x", y)

That is, it sets the same proxy for ALL urllib2 requests. How can I have different proxies for different openers?


Solution

  • Try with pycurl:

    import pycurl
    c1 = pycurl.Curl()
    c1.setopt(pycurl.URL, 'http://www.google.com')
    c1.setopt(pycurl.PROXY, 'localhost')
    c1.setopt(pycurl.PROXYPORT, 8080)
    c1.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5)
    
    c2 = pycurl.Curl()
    c2.setopt(pycurl.URL, 'http://www.yahoo.com')
    c2.setopt(pycurl.PROXY, 'localhost')
    c2.setopt(pycurl.PROXYPORT, 8081)
    c2.setopt(pycurl.PROXYTYPE, pycurl.PROXYTYPE_SOCKS5)
    
    c1.perform() 
    c2.perform()