Python requests session does not rotate proxies

Question

I am using private rotating proxy provided by (https://proxy.webshare.io/proxy/rotating?) in which each request to rotating proxy receives a new IP address. when I am using requests.get('https://httpbin.org/get', headers=headers, proxies=get_proxy()) it returns a new IP each time whenever I make request. but when using

session = requests.Session()
session.headers = headers
session.proxies = get_proxy()

session.get('https://httpbin.org/get')

It returns same IP each time whenever I make request. How does session object behaves different from requests.get() function in case of proxies.

Answer 1

Session uses previously set up variables/values for each subsequent request, like Cookies. If you want to change the proxy for each request in the session, then use Prepared Requests to set it each time or just put it in a function:

def send(session, url):
    return session.get(url, proxy=get_proxy())

sess = requests.Session()
sess.headers = headers
resp = send(sess, 'https://httpbin.org/get')
print(resp.status_code)

But if you're trying to hide your origin IP for scraping or something, you probably don't want to persist cookies, etc. so you shouldn't use sessions.