Search code examples
pythonpython-2.7posthttp-request

http requests : from urllib2 to requests python 2.7

I've been trying to switch all my http requests in a script from using urllib1/2 to using Requests to do more advanced requests.

Clearly, I fail to do so.

Can anyone explain me the difference between :

import urllib,urllib2
data=urllib.urlencode(params)
req=urllib2.Request("http://play.pokemonshowdown.com/action.php",data,{ 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36)' })
response=urllib2.urlopen(req)
response.read()

And :

import requests
r=requests.post("http://play.pokemonshowdown.com/action.php",params=params,headers={ 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36)' })
r.text

?

The first one yields stuff :

']{"curuser":{"userid":"artemisfowl2","usernum":"5717357","username":"Artemis Fowl 2","email":"","registertime":"1444568721","group":"1","banstate":"0","ip":"78.226.235.170","loggedin":true,"outdatedpassword":false},"actionsuccess":true,"assertion":"3f457310c72114c316010110c6dc8f319593d2952b0ebf9a5621b0cdead6db9fce3b7324f158668f82b137135e756b01886df1e297854e536a32166706e93b577ff3a51ee20e19de94fcd7cec4a8668da750d144bbb2cb3fa58fe06fd3d72409f38ed92a28e3631bf0cce73733d0d66b73a147a86e1d7db900c3c46b7de2b429,artemisfowl2,2,1444579736,;2c5deb8c47691071bcd624078e9c7ca1bfa42d7b0ca92109315b7049342206718db2b198fd16f9983049209c23a303ec4dc8bb7a101301ec4603ff19719e4022a3bb7266203c371620407051c94512d038ff6128919c65313d2c006b38f2f0e10ea14870aa2f3132f80904b49a1bed049e0d45000088c95d9ea0b7e34fdd1fa2da4f6f8fb18afa39243c628c8130633a2639ce62ff6b01bd1c562cc009361e2b3edf1a41d96043c32d5cb86c7082cab45c1ced2fd438a12cf8eeb6d0638a4c7f0b9793152bb709f21b79311cfc74c80f1878852ae1b3658197495de98e07b0e3bb9ea949cbc0501fab39cbb13ff7db22077198a61397072be47dacbc7929e8dde5069c2e307dacb57ce41cdc832866aa79c625bfba320699a838bf33d31571e9acb29d846938a232870ae7d42b7689f6b8b03b4318dada3d4aba22e5f46eed0d4a3137bbbcde4c7ebadcb8c42dc5b495f4a5e8e0f4cd18da6013d5d119b5050d2bae9a1281792b83d350b96cd34ebf892116b9bc43d654e30438223315214310c689630b40185f3dc36a557e2cafda3d9272a525f021b56585ca189036649ce8286e13aadd60c076f7efc9003b2819f27e8496130aad405448424fd49720b8faa2661fc139b25ed770edf5eff70fd939eb42d31495edf61b3a0df1328ca95129f7becc5d62b27e70594f45337f633c5e1901499ab822cfb2cda9fa87ee213ca6"}'

While the second does not :

u''

I can confirm the params are correctly encoded in both case and the urls are the same. Also, the request I want to do is a POST request.

So then, why do I get an empty answer ?

Solution

  • I believe that your endpoint is expecting the data to be in the request body as opposed to the query parameters, Try using 

    import requests
r=requests.post("http://play.pokemonshowdown.com/action.php",data=params,headers={ 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36)' })
r.text

    I think that should solve your problem.

    To answer your question:

    urllib2.Request("http://play.pokemonshowdown.com/action.php",data,{ 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36)' })

    adds the data to the body of your request by providing the data argument you implicitly create a post message with the data in the body of the request.

    The line:

    r=requests.post("http://play.pokemonshowdown.com/action.php",params=params,headers={ 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36)' })

    adds the data to the url (like this: www.hostname.com?param1=this&param2=that&param3=theother) instead of to the body of the request. By changing params to data the data should be added to the body as the server is expecting.