Curl request in python

Question

I'm a newbie to python program. I want to get trending topics in google trends. How do i make this curl request from python

curl --data "ajax=1&htd=20131111&pn=p1&htv=l" http://www.google.com/trends/hottrends/hotItems

I tried following code

param = {"data" :"ajax=1&htd=20131111&pn=p1&htv=l"} 
value = urllib.urlencode(param)

req = urllib2.Request("https://www.google.co.in/trends/hottrends/hotItems", value)
response = urllib2.urlopen(req)
result = response.read()
print result

But it is not returning expected values, which is the current Google trends. Any help would be appreciated. Thanks.

Answer 1

You are misinterpreting the data element in your curl command line; that is the already encoded POST body, while you are wrapping it in another data key and encoding again.

Either use just the value (and not encode it again), or put the individual elements in a dictionary and urlencode that:

value = "ajax=1&htd=20131111&pn=p1&htv=l"
req = urllib2.Request("https://www.google.co.in/trends/hottrends/hotItems", value)

or

param = {'ajax': '1', 'htd': '20131111', 'pn': 'p1', 'htv': 'l'}
value = urllib.urlencode(param)
req = urllib2.Request("https://www.google.co.in/trends/hottrends/hotItems", value)

Demo:

>>> import json
>>> import urllib, urllib2
>>> value = "ajax=1&htd=20131111&pn=p1&htv=l"
>>> req = urllib2.Request("https://www.google.co.in/trends/hottrends/hotItems", value)
>>> response = urllib2.urlopen(req)
>>> json.load(response).keys()
[u'trendsByDateList', u'lastPage', u'summaryMessage', u'oldestVisibleDate', u'dataUpdateTime']
>>> param = {'ajax': '1', 'htd': '20131111', 'pn': 'p1', 'htv': 'l'}
>>> value = urllib.urlencode(param)
>>> value
'htv=l&ajax=1&htd=20131111&pn=p1'
>>> req = urllib2.Request("https://www.google.co.in/trends/hottrends/hotItems", value)
>>> response = urllib2.urlopen(req)
>>> json.load(response).keys()
[u'trendsByDateList', u'lastPage', u'summaryMessage', u'oldestVisibleDate', u'dataUpdateTime']