Search code examples
urllib2wgethttp-proxy

http_proxy setting

I know this is simple.. I am jus missing something.. I give up!!

#!/bin/sh
export http_proxy='http://unblocksitesnow.info'
rm -f index.html*
strace -Ff -o /tmp/mm.log -s 200 wget 'http://slashdot.org'

I have used different proxy servers.. to no avail.. I get some default page.. In /etc/wgetrc use_proxy = on

Actually I am trying to use this setting(http_proxy) with python's urllib2. It access some default page as well..

strace - does a dns lookup of the proxy server GET http://slashdot.org/ HTTP/1.0\r\nUser-Agent: Wget/1.11.4\r\nAccept: /\r\nHost: slashdot.org\r\n\r\n

Any pointers??

Solution

  • The problem was I was using proxy sites. These sites expect you to send GET request to the proxy site (with the target site as parameter in URL or whatever site specific mechanisms they implement).

    I was looking for proxy sites like http://www.proxy4free.com/page1.html

    I connect to their respective ports and send a get request to the original target site..