Can't get Google's Search API to work with Python

I'm using google's own search API but I keep getting a 403 error. The key is taken from console.developers.google.com under APIs & auth -> Credentials and I used a Browser key with any referrers. The ID is taken from under the custom search engines basic information.

import requests

search = "https://www.googleapis.com/customsearch/v1"
key = "?key=MY_KEY"
id_ = "&cx=MY_ID"
query = "&q=test"

get = search + key + id_ + query

r = requests.get(get)

print(r)

What am I doing wrong?

Solution

I don't know if this is the source of your problem or not, but you could be making much better use of the requests library. For starters, you can put your API key and CX value into a session object, where they can be used in subsequent requests:

>>> import requests
>>> s = requests.Session()
>>> s.params['key'] = 'MY_KEY'
>>> s.params['cx'] = 'MY_CX'

And you can pass additional search parameters by passing a dict in the params keyword, rather than building the URL yourself:

>>> result = s.get('https://www.googleapis.com/customsearch/v1', 
... params={'q': 'my search string'})

This all works for me:

>>> result
<Response [200]>
>>> print result.text
{
 "kind": "customsearch#search",
 "url": {
  "type": "application/json",
  "template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe
[...]

Also, it's worth checking that you have enabled the search API for your API key.

You can see exactly what the requests library is doing by enabling debug logging through the Python logging module:

>>> import logging
>>> logging.basicConfig(level='DEBUG')
>>> result = s.get('https://www.googleapis.com/customsearch/v1', params={'q': 'openstack'})
DEBUG:requests.packages.urllib3.connectionpool:"GET /customsearch/v1?q=openstack&cx=0123456789123456789%3Aabcdefghijk&key=THIS_IS_MY_KEY HTTP/1.1" 200 13416