Search code examples
httpcurlpostpython-requestswireshark

Python file upload post request not pushed as expected

I am trying to upload an image attachment through an API, the something can be achieved through a web interface. Both make requests to the same url, same headers, everything. But the issue is that the mimetype of the png image is not detected by the server when I make the python request. When retrieving the image again, the content-Type is 'application/octet-stream' rather than 'image/png' (which is what I get when I retrieve the interface-uploaded one). this causes further problems down the road in my pipeline.

Here's a detailed view of the correct (webinterface) request: (via Wireshark) correct request analysis

and the python request: wrong request analysis

The main difference I see here is that in the correct one, the file is detected as 'Portable Network Image' rather than 'Media Type'. I can't find anything related to how it decides that, so to know that would be great.

I have tried using the exact same header set with the correct tokens, still the same problem.

Here's my code for making the request:

headers2 = { \
    'IUCLID6-USER' : headers['USER'] ,
    'IUCLID6-PASS' : headers['PASS'] ,
    'Accept' : 'application/json, text/plain, */*' ,
    'Accept-Encoding': 'gzip, deflate',
    'Connection' : 'keep-alive',
    'Content-Disposition' : "attachment;name="+fname+";filename*=utf-8''" + fname,
    'Content-Length' : str(os.path.getsize(os.path.join(sys.path[0], fname))),
    'Content-Type': 'image/png'
    }

    files = {'attachment': (fname, open(os.path.join(sys.path[0], fname), 'rb'), 'image/png'),}
    data= {'filetype' : 'image/png'}
    # files = {'attachment': open(os.path.join(sys.path[0], fname), 'rb')}
    response = requests.post(base_url + "raw/attachment", headers=headers2, files=files)

Note: when using the data arg rather than files, I noticed that the retrieved 'Content-Type' became 'text/plain'. This is the only case where it changed.

Another Note: I'm not sure if the key in 'files' has to do anything with it but I have tried 'files', 'file', 'media', 'image', and 'data' to no avail. Would be also good to know based on what should the files object be structured.

I don't know why the python request doesn't match the interface request, and its driving me crazy. So any help would be appreciated.

EDIT: the proper request's curl command:

curl 'http://url.com:8080/iuclid6-ext/api/ext/v1/raw/attachment' \
  -X 'POST' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en' \
  -H 'Authorization: Token eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJhNDIzOWI3My1kZDQ0LTRmZGEtOTRkOS1lM2NlZGMyZDZiZjMiLCJpc3MiOiJpdWNsaWQ2LXRva2VuLXNlcnZlciIsInN1YiI6IlN1cGVyVXNlciIsImlhdCI6MTY5MTQwNDQ3MSwiZXhwIjoxNjkxNDA1MDcxLCJzY29wZXMiOlsiYWNjZXNzIl19.VWkb6XYEOKulICaXgLN9gkq4qvdKFDFwwx5b_zeVSlU' \
  -H 'Connection: keep-alive' \
  -H $'Content-Disposition: attachment;filename*=utf-8\'\'Logo-for-a-brand-named-Petrol--style-is-a-mix-of-vintage--retro--sci-fi-space--diesel--motorsport--and-mad-max-styles--The-logo-consists-of-the-name-Petrol-and-a-gas-canister-in-a-stylish-and-pretty-l%20%281%29%20%281%29.png' \
  -H 'Content-Length: 1058156' \
  -H 'Content-Type: image/png' \
  -H 'Origin: http://url.com:8080' \
  -H 'Referer: http://url.com:8080/iuclid6-web/raw/REFERENCE_SUBSTANCE/261bbd71-d826-43ef-9227-93403e892422/(main-content:content/iuclid6:%2F0%2FREFERENCE_SUBSTANCE%2F261bbd71-d826-43ef-9227-93403e892422/)?node=261bbd71-d826-43ef-9227-93403e892422' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44' \
  --compressed \
  --insecure
Solution

  • Upon the suggestion of the awesome @MendelG. I managed to replicate the correct request in Postman. after exporting to python it turns out that the payload should be just the binary file assigned to the data arg as such🤦‍♂️

    data= open(os.path.join(sys.path[0], fname), 'rb')
response = requests.post(base_url + "raw/attachment", headers=headers2, data=data)

    Here's to spending hours debugging a tiny issue that could've been resolved with proper documentation.🥂