Unable to get 'set-cookie' Header in Python requests

Question

# python version: Python 3.11.4
# requests library version: 2.31.0

import requests
import json

# In the actual code I put the real domain instead of example.com
url = "http://api.example.com/auth/sign_in"

headers = {
    "Accept": "application/json, text/plain, */*",
    "Accept-Encoding": "gzip, deflate",
    "Accept-Language": "en-US,en;q=0.7",
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Host": "api.example.com",
    "Origin": "http://www.example.com",
    "Referer": "http://www.example.com/",
    "Sec-Gpc": "1",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"
}

payload = {
    "user_login_id": "admin",
    "user_login_pw": "123123"
}

# At first I tried calling the API with `requests.post(url, headers=headers, data=json.dumps(payload))`
# instead of `session = requests.Session()` and session.post(...)`, but the result was the same.

session = requests.Session()

response = session.post(url, headers=headers, data=json.dumps(payload))

if response.status_code == 200:
    data = json.dumps(dict(response.headers), indent=4)
    print(f"response headers: {data}")
    set_cookie = response.headers.get('set-cookie')
    if set_cookie:
        print(f"Set-Cookie Header: {set_cookie}")
    else:
        print("Set-Cookie Header not found in response.")
else:
    print(f"API request failed. status code: {response.status_code}")

When I ran this code, this was output to the console:

response headers: {
    "x-powered-by": "Express",
    "access-control-allow-origin": "http://www.example.com",
    "vary": "Origin",
    "access-control-allow-credentials": "true",
    "content-type": "application/json; charset=utf-8",
    "content-length": "26",
    "etag": "W/\"1a-pIPrt4esgEyEkX/w62Rnrj9XXdg\"",
    "date": "Sat, 09 Sep 2023 06:33:34 GMT",
    "connection": "close"
}
Set-Cookie Header not found in response.

However, calls to the same API will appear differently in the browser's developer tools. In the network tab, the response headers for this API are displayed as follows.

HTTP/1.1 302 Found
x-powered-by: Express
access-control-allow-origin: http://www.example.com
vary: Origin, Accept
access-control-allow-credentials: true
location: /auth/sign_in_success
content-type: text/plain; charset=utf-8
content-length: 43
set-cookie: connect.sid=s%3Aen3W3Eq0yQs_k7rJZUgWImUAIJKCJNh1.vNS6oiSysS408wodfB%2FBv64IBt7qCaRAfviZw8LQ3Kc; Path=/; HttpOnly
date: Sat, 09 Sep 2023 04:42:01 GMT
connection: close

I need to get information about set-cookie, but I can't see that information in my python code output. How can I achieve this?

After searching, I saw an answer to another stackoverflow question that said to use session instead of requests.post(), so I used session.post(), but I couldn't get any information about set-cookie.

Answer 1

Sorry. It was my stupid mistake.

I was able to get it successfully by just changing the code like this:

import requests
import json

url = "http://api.example.com/auth/sign_in"

# same headers value as previous code
headers = { ... }

# same payload value as previous code
payload = { ... }

session = requests.Session()
response = session.post(url, headers=headers, data=json.dumps(payload))

# just using `session.cookies.get_dict()`
print(f"Set-Cookie Header: {session.cookies.get_dict()}")