Search code examples
pythonweb-scrapinghttp-proxy

Why does the proxy server use my private IP?

I am trying to scrape using proxies (this proxy server is a free one from the internet); in particular I would like to use their IP, not my private one. To test my script I am trying to Access "http://whatismyipaddress.com/" to see which IP this site sees. As it turns out it will see my private IP. Can somebody tell me what's wrong here?

import requests
from fake_useragent import UserAgent

def getMyIP(proxyServer,myPrivateIP):

    scrape_website = "http://whatismyipaddress.com/"

    ua = UserAgent()
    headers = {'User-Agent': ua.random}
    
    try:
        response = requests.get(scrape_website,headers=headers,proxies={"https":proxyServer})
    except:
        faultString = proxyServer + " did not work; " + "\n" 
        print(faultString)
        return
    
    if myPrivateIP in str(response.content):
        print("They found my private IP.")

proxyServer = "http://103.250.158.23:61219"
myPrivateIP = "xxx.xxx.xxx.xxx"
getMyIP(proxyServer,myPrivateIP)
Solution

  • Two things:

    1. You set an {'https': ...} proxy configuration. This means for any HTTPS requests, it will use that proxy. You're requesting an HTTP URL however, so that proxy isn't getting used. Configure an 'http' proxy instead or in addition.
    2. If the proxy forwards your IP in an HTTP header, and the target server heeds that header, that's tough luck and nothing you can do anything about, besides using a different proxy which doesn't forward your IP. I think point 1 is more likely the issue though.