Search code examples
pythonapiweb-scrapingrequestinsomnia

Using API-Endpoint from website allways get invalid response?


i try to use the api-endpoint from this site: https://horoguides.com/hk/watch_finder

I searched for the api-endpoint in the network-tab and try to rebuild this api-access with the following code:

import requests

url = "https://horoguides.com/hk/ajaj/watch/searchWatches"
payload = {
    "addLimit": "LIMIT 0, 20",
    "addOrder": "ORDER BY establish DESC",
}
headers = {
    'Accept': "application/json, text/javascript, */*; q=0.01",
    'Accept-Language': "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7",
    'Connection': "keep-alive",
    'Content-Type': "multipart/form-data; boundary=---011000010111000001101001",
    'Cookie': "PHPSESSID=siob5k70qu4gh8bkio07qtocv3; _gid=GA1.2.40295814.1663575664; __gads=ID=2fc582d62ff2a986-223e4e8c26ce00a9:T=1663575664:RT=1663575664:S=ALNI_MaTX_1U4CELXasmH0td3MvCRQ5S5Q; _gat_UA-90322481-1=1; _gat_gtag_UA_90322481_1=1; _ga_6Z9E9PKG02=GS1.1.1663594500.3.1.1663594710.0.0.0; _ga=GA1.1.699639573.1663575664",
    'Origin': "https://horoguides.com",
    'Referer': "https://horoguides.com/hk/watch_finder",
    'Sec-Fetch-Dest': "empty",
    'Sec-Fetch-Mode': "cors",
    'Sec-Fetch-Site': "same-origin",
    'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36",
    'X-Requested-With': "XMLHttpRequest",
    'sec-ch-ua': "^\^Chromium^^;v=^\^104^^, ^\^"
    }
resp = requests.request("POST", url, json=payload, headers=headers)
print(resp.status_code)
respJSON = resp.json()
print(respJSON)

But as response i only get:

200
{'status': 'invalid'}

Why is this reponse from the api-endpoint not working? I also tried to run this in Insomnia and get the same result.


Solution

  • You need to fix the payload. The following code works:

    import requests
    
    url = "https://horoguides.com/hk/ajaj/watch/searchWatches"
    payload = {
        "addLimit": "LIMIT 0, 20",
        "addOrder": "ORDER BY establish DESC",
        'lang': 'hk',
        'ajaxID': 'searchWatches'
    }
    headers = {
        'Accept': "application/json, text/javascript, */*; q=0.01",
        'Accept-Language': "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7",
        'Connection': "keep-alive",
        'Origin': "https://horoguides.com",
        'Referer': "https://horoguides.com/hk/watch_finder",
        'Sec-Fetch-Dest': "empty",
        'Sec-Fetch-Mode': "cors",
        'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36",
        'X-Requested-With': "XMLHttpRequest"
        }
    resp = requests.request("POST", url, data=payload, headers=headers)
    print(resp.status_code)
    respJSON = resp.json()
    print(respJSON)
    

    Result in terminal:

    200
    {'act': 'watch/searchWatches', 'status': 'success', 'getData': {'a5124': {'id': '5124', 'name': '116610-LN-0001', 'url_name': '116610-ln-97200', 'establish': '2014', 'w_brand_id': '39', 'w_brand_abbr': '', 'w_brand_name': 'ROLEX', 'w_brand_urlname': 'rolex', 'w_brand_localname': '勞力士', 'hype_default_currency': 'NT$', 'w_series_name': 'SUBMARINER', 'w_series_urlname':[....]
    

    For requests documentation, see https://requests.readthedocs.io/en/latest/