Why a curl POST request is returning a different result than this Common Library POST request to the same address with the same body?

Question

I am using Steel Bank Common Lisp (SBCL), Emacs, Slime, and a library called Dexador.

In the REPL, I can do:

CL-USER> (dex:post "https://httpbin.org/post"
          :content "{\"user\": \"user1\", \"pass\":\"abcd\"}")
"{
  \"args\": {}, 
  \"data\": \"{\\\"user\\\": \\\"user1\\\", \\\"pass\\\": \\\"abcd\\\"}\", 
  \"files\": {}, 
  \"form\": {}, 
  \"headers\": {
    \"Accept\": \"*/*\", 
    \"Content-Length\": \"33\", 
    \"Content-Type\": \"text/plain\", 
    \"Host\": \"httpbin.org\", 
    \"User-Agent\": \"Dexador/0.9.15 (SBCL 2.1.9.nixos); Linux; 5.10.94\", 
    \"X-Amzn-Trace-Id\": \"Root=1-62956dc4-1b95d37752a67b8420180f71\"
  }, 
  \"json\": {
    \"pass\": \"abcd\", 
    \"user\": \"user1\"
  }, 
  \"origin\": \"189.2.84.243\", 
  \"url\": \"https://httpbin.org/post\"
}
"

Removing the escape character, notice the JSON key of the JSON output:

  "json": {
    "pass": "abcd", 
    "user": "user1"
  }

Now, if I try doing the same on curl following this tutorial, this is what I get:

$ curl -d "user=user1&pass=abcd" -X POST https://httpbin.org/post

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "pass": "abcd", 
    "user": "user1"
  }, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "20", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.79.1", 
    "X-Amzn-Trace-Id": "Root=1-62956e50-4e880d101ac23b781fe0f9d5"
  }, 
  "json": null, 
  "origin": "189.2.84.243", 
  "url": "https://httpbin.org/post"
}

Notice the "json": null,.

I was expecting both results to be the same since they are pointing to the same address and have the same body being sent.

Why are they different? Did I miss something?

Answer 1

Set up netcat:

nc -l -p 3500

Run your curl command against that:

curl -d "user=user1&pass=abcd" -X POST http://localhost:3500/post

Netcat prints:

POST /post HTTP/1.1
Host: localhost:3500
User-Agent: curl/7.68.0
Accept: */*
Content-Length: 20
Content-Type: application/x-www-form-urlencoded

user=user1&pass=abcd

So first, Content-Type is application/x-www-form-urlencoded, because that's what -d means to curl.

Second, the data is sent as user=user1&pass=abcd, not as JSON. That's because curl's default data mode is application/x-www-form-urlencoded, it doesn't just send it as JSON because you hoped it would.
In fact, it doesn't implement JSON at all.

If you want to send JSON, you have to do that explicitly:

jq --null-input '
    .user |= "user1" | 
    .pass |= "abcd"' \
    | curl \
        -H 'Content-Type: text/plain' \
        -d @- \
        -X POST \
        https://httpbin.org/post

Response:

{
  "args": {}, 
  "data": "{  \"user\": \"user1\",  \"pass\": \"abcd\"}", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "36", 
    "Content-Type": "text/plain", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/7.68.0", 
    "X-Amzn-Trace-Id": "Root=1-6295a78d-0d828c3c6d3891174fd22d01"
  }, 
  "json": {
    "pass": "abcd", 
    "user": "user1"
  }, 
  "origin": "71.122.242.250", 
  "url": "https://httpbin.org/post"
}