Search code examples
pythonjsonweb-scrapingcloud

How to I make an API request in Python


I am trying to make an API request to my online web site scraper at CloudScrape, the documentation shows the following example.

POST /api/runs/{runId}/execute/inputs/wait HTTP/1.1

X-CloudScrape-Access: ga09usdm8osdf8n9sodf
X-CloudScrape-Account: 87750c5f-7423-4438-b3f1-7b8b86990621
Accept: application/json
Accept-Encoding: gzip
Content-Type: application/json
Host: app.cloudscrape.com
User-Agent: YourApp/1.0 >
> { > "my_input_field": "My input value", > "my_other_input_field": 234 > }

I am trying to post the following parameters;

body, runId

So far, I have tried

import requests
import json

client = requests.session()

runID = '86df246f-8f77-47b9-9655-be61e3851839'
inputQuery = {"query": "1080p category:movies user:z0n321 imdb:3659388"}
data = {'body':inputQuery, 'runId':'86df246f-8f77-47b9-9655-be61e3851839'}
headers = {'X-CloudScrape-Access':'a41c493c2a31d068f1a06333311e52f7', 'X-CloudScrape-Account':'87750c5f-7423-4438-b3f1-7b8b86990621', 'Accept':'application/json', 'Content-Type':'application/json'}

response = client.post('https://app.cloudscrape.com/api/runs/{%s}/execute/inputs/wait' % runID, data=json.dumps(data), headers=headers)

results = json.loads(response.content)
print results

I get this response {u'code': 404, u'error': True}

My scraper runs fine when I run it manually through their website. Am I doing something wrong?


Solution

  • You don't need the braces around the interpolated parameter.

    ...api/runs/%s/execute...