Error 500 when requesting Datadog logs by Python Requests

Question

I have the following curl command which works fine:

curl -X POST -H 'content-type: application/json' -H "DD-API-KEY: ${api_key}" -H "DD-APPLICATION-KEY: ${app_key}" \
-d '{ 
        "query": "service:my_service",
        "time": {
            "from": "2019-11-28T00:00:00Z",
            "to": "2019-11-28T16:00:00Z"
        },
        "sort": "asc",
        "limit": 1000
    }' "https://api.datadoghq.com/api/v1/logs-queries/list" -o output3.json5

Then I convert this requests to Python Requests, and the curl method works but Python returns a 500 error without any details.

import requests

def main():
    headers = {
        'content-type': 'application/json',
        'DD-API-KEY': 'AAA',
        'DD-APPLICATION-KEY': 'XXX',
    }

    data = {
        "query": "service:my_service",
        "time": {
            "from": "now - 1h",
            "to": "now"
        },
        "sort": "asc",
        "limit": 50
    }
    response=requests.post("https://api.datadoghq.com/api/v1/logs-queries/list",headers=headers, data=data)

I tried it outside my Docker guessing that maybe connection was the key, but it doesn't work either.

Answer 1

Point both of those at a service like httpbin to see how they differ.

Requests' data option for POST requests generates form-encoded data by default, while curl passes the JSON string through directly. You can manually encode your payload as a JSON string:

import json

response = requests.post(..., data=json.dumps(data))
#                                  ^^^^^^^^^^

or if you have Requests version 2.4.2 or later you can use the json parameter to have your dict converted to JSON automatically:

response = requests.post(..., json=data)
#                             ^^^^