Search code examples
pythonhtmlpython-requestspostman

Python request working on postman but not in local machine


I am scraping using postman and I have located what data I need I just need it to work locally. Although it works in postman and in kaggle but not in local machine neither in colab. Here's the code for reference: `

import requests

url = "https://apisae1.phdvasia.com/v1/product-hut-fe/localizations?order_type=C&disposition=C&limit=4"

payload = {}
headers = {
  'authority': 'apisae1.phdvasia.com',
  'accept': 'application/json, text/plain, */*',
  'accept-language': 'en-US,en;q=0.9',
  'client': '57950680-0208-4703-9e0b-c5a8cc45cf42',
  'lang': 'pt',
  'origin': 'https://www.pizzahut.com.br',
  'referer': 'https://www.pizzahut.com.br/',
  'sec-ch-ua': '"Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"',
  'sec-ch-ua-mobile': '?0',
  'sec-ch-ua-platform': '"Windows"',
  'sec-fetch-dest': 'empty',
  'sec-fetch-mode': 'cors',
  'sec-fetch-site': 'cross-site',
  'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 OPR/99.0.0.0',
  'Cookie': 'ak_bmsc=CA52AD92D65BE36761FAC7C181F253D0~000000000000000000000000000000~YAAQxGvcF7RDj0SJAQAALPEpSRSGE7kXTAwqrRr6W7FKHAmAiIZLed4usMa8hURw56blJZSho5ANdgoKbhEiJCGu8lFXst2L2uWHVaoNXi3QZgkcGHgYmHvWNw94pK9aoh4zr9YvHaeMFRd5Xr01mSDpSfjtx7PwGydeGkLcRo+2PX6xlXM9UPSDal59DPOjkM0LclJgl0s3FOumelzJrL5f73VPNQBEmh9Ek+XInsBiO5z6x118opcFaDNoRGAW/8rnaYJgQNF/t7WgfddKr7VA6Dif7s6mA0kxiUYlXpjQLuaFvCNc++RO6+fqm0fPz8bz8BQPKg9OIjzDPSdXUrRMkpQhdmImyzNDv/v2MePK8H2BYZnjh1JK0Kuw; bm_sv=FBC11E925F19B3C0A6BA6708D42A3D3A~YAAQxGvcF8yxkESJAQAAFkxASRT933k/B3sEomiSPBAzN4AEskTpDt7ilDa1EZwRgSq0vPW3eeF1pcmF8mfeEzeUTd9M9cLkVh3FoENhHyaorUh9yPYxu06HHCdXltInBWkY5jXsNa18804DF6tVDieU0ZZd4cKm8o1AMB1luUPGsLPJewgPVQWNWMeKAz6PyZVo+qAcZPW67ACmEzr4J+SONmXPAiPlK0jn77bU8PEOWAukDLbeO6BzwGFm6a4ESjr3~1'
}

response = requests.request("GET", url, headers=headers, data=payload, allow_redirects=False)

print(response.text)

I tried changing user agent to random and even current version of browser I'm using and tried on other cloud services. It worked on kaggle but not on colaband local machine. PLs help


Solution

  • The only header value you need is the client.

    import requests
    import json # only used for presentation
    
    url = "https://apisae1.phdvasia.com/v1/product-hut-fe/localizations?order_type=C&disposition=C&limit=4"
    
    headers = {
        'client': '57950680-0208-4703-9e0b-c5a8cc45cf42'
    }
    
    with requests.get(url, headers=headers) as response:
        response.raise_for_status()
        print(json.dumps(response.json(), indent=2))
    

    Without the client header (or an invalid client) you'll get HTTP 400