Why can't I POST to Django with pyCurl?

Question

I've hit something truly strange with a pyCurl script hitting a local Django-Tastypie REST webserver.

Issuing HTTP PUT requests to the server succeeds when I use everything but pycurl (including curl), and fails with error 400 in pycurl.

After much googling and experimentation, I'm stumped. What's wrong here?

Curl call that works:

curl --verbose -X PUT -H 'Content-Type: application/json' -d '{"first_name": "Gaius","id": 1,"last_name": "Balthazar","login": "gbalthazar"}' http://localhost:8000/api/person/1/

PyCurl call that DOESN'T work (error 400):

import pycurl
import StringIO
curl = pycurl.Curl()
url = 'http://localhost:8000/api/person/1/'
curl.setopt(pycurl.URL,url)
curl.setopt(pycurl.VERBOSE, 1)
body = '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
curl.setopt(pycurl.READFUNCTION, StringIO.StringIO(body).read)
curl.setopt(pycurl.UPLOAD, 1)
curl.setopt(pycurl.HTTPHEADER,['Content-Type: application/json','Expect:'])
curl.setopt(curl.TIMEOUT, 5)
curl.perform()

(I've tried removing the Expects header as well, I see the header set to 100-Continue in the pycurl call, but same result.)

Unfortunately this project really does need pycurl's low-level access to HTTP timing stats to measure performance, so I can't do it with another HTTP/REST library.

Output of Curl Call:

* About to connect() to localhost port 8000 (#0)
*   Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
> User-Agent: curl/7.27.0
> Host: localhost:8000
> Accept: */*
> Content-Type: application/json
> Content-Length: 78
> 
* upload completely sent off: 78 out of 78 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Thu, 05 Jun 2014 23:45:26 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< Vary: Accept
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
< 
* Closing connection #0
{"first_name": "Gaius", "id": 1, "last_name": "Balthazar", "login": "gbalthazar", "pk": "1", "resource_uri": "/api/person/1/"}

Output of PyCurl Verbose Call:

* About to connect() to localhost port 8000 (#0)
*   Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
User-Agent: PycURL/7.27.0
Host: localhost:8000
Accept: */*
Transfer-Encoding: chunked
Content-Type: application/json

* HTTP 1.0, assume close after body
< HTTP/1.0 400 BAD REQUEST
< Date: Thu, 05 Jun 2014 23:44:25 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
< 
* Closing connection #0
{"error": ""}

What am I missing here?

Answer 1

Found the answer: It needs length of the request body to handle correctly

For POST:

curl.setopt(pycurl.POSTFIELDSIZE, len(body))  

For PUT:

curl.setopt(pycurl.INFILESIZE, len(body))

(Yes, it's a different option for different HTTP calls... that's libcurl for you)

Not completely sure what triggers this behaviour, but the above fixes it and the tests work now.

EDIT: Adding verbose pycurl output from this:

* About to connect() to localhost port 8000 (#0)
*   Trying 127.0.0.1...
* connected
* Connected to localhost (127.0.0.1) port 8000 (#0)
> PUT /api/person/1/ HTTP/1.1
User-Agent: PycURL/7.27.0
Host: localhost:8000
Accept: */*
Content-Type: application/json
Content-Length: 72

* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Fri, 06 Jun 2014 17:41:38 GMT
< Server: WSGIServer/0.1 Python/2.7.3
< Vary: Accept
< X-Frame-Options: SAMEORIGIN
< Content-Type: application/json
< 
* Closing connection #0
{"first_name": "Gaius", "id": 1, "last_name": "Baltar", "login": "gbaltar", "pk": "1", "resource_uri": "/api/person/1/"}