Search code examples
pythonpython-requestsconfluence-rest-api

python requests.put() fails when urllib3 http.request('PUT', ...) succeeds. What gives?


I am trying to hit the Atlassian Confluence REST API using python requests.

I've successfully called a GET api, but when I call the PUT to update a confluence page, it returns 200, but didn't update the page.

I used chrome::YARC to verify that the API was working properly (which it was). After a while trying to debug it, I reverted to try using urllib3, which worked just fine.

I'd really like to use requests, but I can't for the life of me figure this one out after hours and hours of trying to debug, Google, etc.

I'm running Mac/Python3:

$ uname -a
Darwin mylaptop.local 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64
$ python3 --version
Python 3.6.1

Here's my code that shows all three ways I'm trying this (two requests and one urllib3):

def update(self, spaceKey, pageTitle, newContent, contentType='storage'):
    if contentType not in ('storage', 'wiki', 'plain'):
        raise ValueError("Invalid contentType={}".format(contentType))

    # Get current page info
    self._refreshPage(spaceKey, pageTitle) # I retrieve it before I update it.
    orig_version = self.version

    # Content already same as requested content. Do nothing
    if self.wiki == newContent:
        return

    data_dict = {
        'type' : 'page',
        'version' : {'number' : self.version + 1},
        'body'  : {
            contentType : {
                'representation' : contentType,
                'value' : str(newContent)
            }
        }
    }
    data_json = json.dumps(data_dict).encode('utf-8')

    put = 'urllib3' #for now until I figure out why requests.put() doesn't work
    enable_http_logging()
    if put == 'requests':
        r = self._cs.api.content(self.id).PUT(json=data_dict)
        r.raise_for_status()
    elif put == 'urllib3':
        urllib3.disable_warnings() # I know, you can quit your whining now!!!
        headers = { 'Content-Type' : 'application/json;charset=utf-8' }
        auth_header = urllib3.util.make_headers(basic_auth=":".join(self._cs.session.auth))
        headers = {**headers, **auth_header}
        http = urllib3.PoolManager()
        r = http.request('PUT', str(self._cs.api.content(self.id)), body=data_json, headers=headers)
    else:
        raise ValueError("Huh? Unknown put type: {}".format(put))
    enable_http_logging(False)

    # Verify page was updated
    self._refreshPage(spaceKey, pageTitle) # Check for changes
    if self.version != orig_version + 1:
        raise RuntimeError("Page not updated. Still at version {}".format(self.version))
    if self.wiki != newContent:
        raise RuntimeError("Page version updated, but not content.")

Any help would be great.

Update 1: Adding request dump

-----------START-----------
PUT http://confluence.myco.com/rest/api/content/101904815
User-Agent: python-requests/2.18.4
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 141
Content-Type: application/json
Authorization: Basic <auth-token-here>==

b'{"type": "page", "version": {"number": 17}, "body": {"storage": {"representation": "storage", "value": "new body here version  version 17"}}}'

Solution

  • requests never went back to PUT (Bug???)

    What you're observing is requests behaving consistently with web browsers: reacting to HTTP 302 redirect with a GET request.

    From Wikipedia:

    The user agent (e.g. a web browser) is invited by a response with this code to make a second, otherwise identical, request to the new URL specified in the location field.

    (...)

    Many web browsers implemented this code in a manner that violated this standard, changing the request type of the new request to GET, regardless of the type employed in the original request (e.g. POST)

    (...)

    As a consequence, the update of RFC 2616 changes the definition to allow user agents to rewrite POST to GET.

    So this behaviour is consistent with RFC 2616. I don't think we can say which of the two libraries behaves "more correctly".