Search code examples
pythonurllib2urllib

How to send POST request with no data


Is it possible using urllib or urllib2 to send no data with a POST request? Sounds odd, but the API I am using sends blank data in the POST request.

I've tried the following, but it seems to be issuing a GET request because of no POST data.

url = 'https://site.com/registerclaim?cid=' + int(cid)
values = {}
headers = {
    'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
    'X-CRFS-Token' : csrfToken,
    'X-Requested-With' : 'XMLHttpRequest'   
}

data    = urllib.urlencode(values)
req     = urllib2.Request(url, data, headers)
resp    = opener.open(req)

I am getting a 404 exception, which is what the API returns if trying to access the page via a 'GET' request. I've checked all the variables to ensure they are set correctly, and they are.

Am advice?


Solution

  • Your diagnosis is incorrect; urllib2 will send an empty POST body in your case:

    >>> import urllib, urllib2
    >>> url = 'http://httpbin.org/post?cid=42'
    >>> headers = {
    ...     'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
    ...     'X-CRFS-Token': 'csrf_token_mocked',
    ...     'X-Requested-With' : 'XMLHttpRequest'   
    ... }
    >>> values = {}
    >>> data    = urllib.urlencode(values)
    >>> req     = urllib2.Request(url, data, headers)
    >>> req.has_data()
    True
    >>> req.get_method()
    'POST'
    >>> resp    = urllib2.urlopen(req)
    >>> body = resp.read()
    >>> print body
    {"args": {"cid": "42"}, "data": "", "files": {}, "form": {}, "headers": {"Accept-Encoding": "identity", "Connection": "close", "Content-Length": "0", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36", "X-Crfs-Token": "csrf_token_mocked", "X-Request-Id": "a14f84f5-a355-4b8a-8b34-cb42808b8b09", "X-Requested-With": "XMLHttpRequest"}, "json": null, "origin": "81.134.152.4", "url": "http://httpbin.org/post?cid=42"}
    >>> from pprint import pprint
    >>> import json
    >>> pprint(json.loads(body))
    {u'args': {u'cid': u'42'},
     u'data': u'',
     u'files': {},
     u'form': {},
     u'headers': {u'Accept-Encoding': u'identity',
                  u'Connection': u'close',
                  u'Content-Length': u'0',
                  u'Content-Type': u'application/x-www-form-urlencoded',
                  u'Host': u'httpbin.org',
                  u'User-Agent': u'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
                  u'X-Crfs-Token': u'csrf_token_mocked',
                  u'X-Request-Id': u'a14f84f5-a355-4b8a-8b34-cb42808b8b09',
                  u'X-Requested-With': u'XMLHttpRequest'},
     u'json': None,
     u'origin': u'81.134.152.4',
     u'url': u'http://httpbin.org/post?cid=42'}
    

    Some things to note:

    • the http://httpbin.org/post route only accepts POST methods; it'll return a 405 response otherwise.
    • The req.get_method() method is used to determine what method to use when sending the request; you can see that it'll use POST even though values is empty. req.get_method() determines the method used based on the req.has_data() response, and that method returns True when the data attribute is not None. An empty string qualifies here as having data.
    • The http://httpbin.org/post response shows the request headers; the content-length header was set to 0, indicating an empty POST body.

    So the question then is: how certain are you you have everything required for the POST to succeed? Perhaps a Referer header is required, or perhaps you misunderstood what parameters are passed in and the POST body is not meant to be empty.