Search code examples
pythonclojurepython-requestsringhttp-kit

Timeout with Python Requests + Clojure HttpKit Server but not Ring server

I have some Ring routes which I'm running one of two ways.

  1. lein ring server, with the lein-ring plugin
  2. using org.httpkit.server, like (hs/run-server app {:port 3000}))

It's a web app (being consumed by an Angular.js browser client).

I have some API tests written in Python using the Requests library:

my_r = requests.post(MY_ROUTE,
                     data=MY_DATA,
                     headers={"Content-Type": "application/json"},
                     timeout=10)

When I use lein ring server, this request works fine in the JS client and the Python tests.

When I use httpkit, this works fine in the JS client but the Python client times out with

socket.timeout: timed out

I can't figure out why the Python client is timing out. It happens with httpkit but not with lein-ring, so I can only assume that the cause is related to the difference.

  • I've looked at the traffic in WireShark and both look like they give the correct response. Both have the same Content-Length field (15 bytes).
  • I've raised the number of threads to 10 (shouldn't need to) and no change.

Any ideas what's wrong?

Solution

  • I found how to fix this, but no satisfactory explanation.

    I was using wrap-json-response Ring middleware to take a HashMap and convert it to JSON. I switched to doing my own conversion in my handler with json/write-str, and this fixes it.

    At a guess it might be something to do with the server handling output buffering, but that's speculation.

    I've combed through the Wireshark dumps and I can't see any relevant differences between the two. The sent Content-Length fields are identical. The 'bytes in flight' differ, at 518 and 524.

    No clue as to why the web browser was happy with this but Python Requests wasn't, and whether or this is a bug in Requests, httpkit, ring-middleware-format or my own code.