Python requests无法获取set-cookie响应头问题求助

使用Python 3.11.4及requests 2.31.0编写代码调用http://api.example.com/auth/sign_in接口，代码如下：

# python version: Python 3.11.4
# requests library version: 2.31.0

import requests
import json

# In the actual code I put the real domain instead of example.com
url = "http://api.example.com/auth/sign_in"

headers = {
    "Accept": "application/json, text/plain, */*",
    "Accept-Encoding": "gzip, deflate",
    "Accept-Language": "en-US,en;q=0.7",
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Host": "api.example.com",
    "Origin": "http://www.example.com",
    "Referer": "http://www.example.com/",
    "Sec-Gpc": "1",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"
}

payload = {
    "user_login_id": "admin",
    "user_login_pw": "123123"
}

# At first I tried calling the API with `requests.post(url, headers=headers, data=json.dumps(payload))`
# instead of `session = requests.Session()` and session.post(...), but the result was the same.

session = requests.Session()

response = session.post(url, headers=headers, data=json.dumps(payload))

if response.status_code == 200:
    data = json.dumps(dict(response.headers), indent=4)
    print(f"response headers: {data}")
    set_cookie = response.headers.get('set-cookie')
    if set_cookie:
        print(f"Set-Cookie Header: {set_cookie}")
    else:
        print("Set-Cookie Header not found in response.")
else:
    print(f"API request failed. status code: {response.status_code}")

运行代码后控制台输出：

response headers: {
    "x-powered-by": "Express",
    "access-control-allow-origin": "http://www.example.com",
    "vary": "Origin",
    "access-control-allow-credentials": "true",
    "content-type": "application/json; charset=utf-8",
    "content-length": "26",
    "etag": "W/\"1a-pIPrt4esgEyEkX/w62Rnrj9XXdg\"",
    "date": "Sat, 09 Sep 2023 06:33:34 GMT",
    "connection": "close"
}
Set-Cookie Header not found in response.

但浏览器开发者工具中调用同一接口的响应头包含Set-Cookie：

HTTP/1.1 302 Found
x-powered-by: Express
access-control-allow-origin: http://www.example.com
vary: Origin, Accept
access-control-allow-credentials: true
location: /auth/sign_in_success
content-type: text/plain; charset=utf-8
content-length: 43
set-cookie: connect.sid=s%3Aen3W3Eq0yQs_k7rJZUgWImUAIJKCJNh1.vNS6oiSysS408wodfB%2FBv64IBt7qCaRAfviZw8LQ3Kc; Path=/; HttpOnly
date: Sat, 09 Sep 2023 04:42:01 GMT
connection: close

需要获取Set-Cookie信息，但Python代码输出中无法看到，尝试用Session替代直接requests.post也未解决，如何实现？

核心原因 #

requests默认会自动跟随HTTP 302重定向，浏览器中看到的是第一个302响应（包含Set-Cookie），但Python代码中获取的response是重定向后的最终200响应，这个响应没有Set-Cookie，因此无法提取到目标信息。

解决步骤 #

1. 禁用自动重定向：在post请求中添加allow_redirects=False参数，直接获取原始的302响应。
2. 提取Cookie：从302响应头中直接读取Set-Cookie，或通过session.cookies获取已保存的Cookie（session会自动存储响应中的Cookie）。

修改后的代码示例：

import requests
import json

url = "http://api.example.com/auth/sign_in"

headers = {
    "Accept": "application/json, text/plain, */*",
    "Accept-Encoding": "gzip, deflate",
    "Accept-Language": "en-US,en;q=0.7",
    "Connection": "keep-alive",
    "Content-Type": "application/json",
    "Host": "api.example.com",
    "Origin": "http://www.example.com",
    "Referer": "http://www.example.com/",
    "Sec-Gpc": "1",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36"
}

payload = {
    "user_login_id": "admin",
    "user_login_pw": "123123"
}

session = requests.Session()

# 禁用自动重定向，获取原始302响应
response = session.post(url, headers=headers, data=json.dumps(payload), allow_redirects=False)

if response.status_code == 302:
    data = json.dumps(dict(response.headers), indent=4)
    print(f"response headers: {data}")
    # 获取单个Set-Cookie值
    set_cookie = response.headers.get('set-cookie')
    if set_cookie:
        print(f"Set-Cookie Header: {set_cookie}")
    # 获取所有Set-Cookie值（若有多个）
    all_set_cookies = response.headers.getlist('set-cookie')
    print(f"All Set-Cookie Headers: {all_set_cookies}")
    # 从session中获取已保存的Cookie字典
    print("Session Cookies:", session.cookies.get_dict())
else:
    print(f"API request failed. status code: {response.status_code}")

额外说明 #

• 如果后续需要访问重定向后的页面，无需手动携带Cookie，直接使用已保存Cookie的session调用session.get(location_url)即可。
• 若Set-Cookie包含多个值，response.headers.get('set-cookie')仅返回第一个，使用response.headers.getlist('set-cookie')可获取全部。

内容的提问来源于stack exchange，提问作者user20231989