使用https.client发起POST请求遇FileNotFoundError及409异常求助

问题描述 #

使用Python https.client模块向外部API发起POST请求时，仅在API返回409响应码时触发SSL异常，200和400响应时请求正常。程序未能进入409分支处理重试逻辑，直接抛出异常。

报错信息 #

Traceback (most recent call last):
  File "z:\this\is\the\path\to\my\script.py", line 95, in update_ag
    conn.request("POST", f"/path/to/api?{req_data}",
  File "C:\Python311\Lib\http\client.py", line 1283, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1329, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1278, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Python311\Lib\http\client.py", line 1038, in _send_output
    self.send(msg)
  File "C:\Python311\Lib\http\client.py", line 976, in send
    self.connect()
  File "C:\Python311\Lib\http\client.py", line 1455, in connect
    self.sock = self._context.wrap_socket(self.sock,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\ssl.py", line 517, in wrap_socket

原请求代码 #

import http.client
import time

conn = http.client.HTTPSConnection("external.api.com")
bad_req = []

def update_ag(dwr, qses, data):
    payload = ''
    cookie = dwr + '; ' + qses
    headers = {
        'X-Requested-With': 'Curl',
        'Authorization': 'xxxxx',
        'Cookie': cookie
    }
    try:
        for i in range(len(data)):
            call = 1
            # 预处理逻辑，生成req_data
            req_data = "example_param=value"  # 示例参数
            if call: # if call is 1, make the request
                while True:
                    conn.request("POST", f"/path/to/api?{req_data}",
                                payload,
                                headers)
                    res = conn.getresponse()
                    res_code = res.getcode()
                    if res_code == 200:
                        res = res.read()
                        x = res.decode("utf-8") 
                        break
                    elif res_code == 409:
                        # 从未进入该分支
                        print("Got 409")
                        res = res.read()
                        x = res.decode("utf-8") 
                        print("Got 409, resending the request in 20 seconds.")
                        time.sleep(20)
                    else:
                        print(f"Request was unsuccessful with error: {res_code}")
                        res = res.read()
                        x = res.decode("utf-8")
                        bad_req.append(req_data + '\n')
                        break
            time.sleep(0.1)
        print("Updating asset groups, finished.")
    except Exception as e:
        print("ERROR - Occured in update_ag():")
        print(e)
        raise RuntimeError("unable to handle error")

运行环境：Azure Pipelines，API繁忙时返回409，预期20秒后重试，但触发SSL异常。

错误原因 #

1. 长连接复用失效：HTTPSConnection默认使用长连接，当API返回409后可能主动关闭了连接，而客户端仍在复用这个已失效的连接套接字。再次发起请求时，尝试复用失效连接会触发SSL握手失败，此时程序还未走到读取响应码的步骤就抛出异常，因此无法进入409分支。
2. 未正确清理连接资源：收到409响应后，没有关闭当前连接并重置连接状态，导致后续请求复用异常连接。

解决方法 #

方法1：每次重试新建连接 #

将连接创建逻辑放到重试循环内部，确保每次请求都使用全新的连接，避免复用失效连接：

import http.client
import time

bad_req = []

def update_ag(dwr, qses, data):
    payload = ''
    cookie = dwr + '; ' + qses
    headers = {
        'X-Requested-With': 'Curl',
        'Authorization': 'xxxxx',
        'Cookie': cookie
    }
    try:
        for i in range(len(data)):
            call = 1
            # 预处理逻辑，生成req_data
            req_data = "example_param=value"  # 示例参数
            if call:
                while True:
                    # 每次重试都新建HTTPS连接
                    conn = http.client.HTTPSConnection("external.api.com")
                    try:
                        conn.request("POST", f"/path/to/api?{req_data}", payload, headers)
                        res = conn.getresponse()
                        res_code = res.getcode()
                        res_content = res.read().decode("utf-8")
                        
                        if res_code == 200:
                            print(f"请求成功: {res_content}")
                            break
                        elif res_code == 409:
                            print(f"Got 409: {res_content}")
                            print("20秒后重试...")
                            time.sleep(20)
                        else:
                            print(f"请求失败，状态码: {res_code}, 响应: {res_content}")
                            bad_req.append(req_data + '\n')
                            break
                    finally:
                        # 确保连接关闭，释放资源
                        conn.close()
            time.sleep(0.1)
        print("Updating asset groups, finished.")
    except Exception as e:
        print(f"ERROR - Occured in update_ag(): {e}")
        raise RuntimeError("unable to handle error")

方法2：复用连接但重置状态 #

如果要保留长连接，在收到409后显式关闭当前连接，下次请求前重新建立连接：

# 在409分支中添加连接关闭逻辑
elif res_code == 409:
    print("Got 409")
    res_content = res.read().decode("utf-8") 
    print(f"Got 409, resending the request in 20 seconds. Response: {res_content}")
    # 关闭当前失效连接
    conn.close()
    # 重新创建连接
    conn = http.client.HTTPSConnection("external.api.com")
    time.sleep(20)

SSL配置说明 #

Azure Pipelines环境默认SSL配置可满足大部分场景，若API有特殊证书要求，可自定义SSL上下文：

import ssl

# 创建默认SSL上下文
context = ssl.create_default_context()
# 若需忽略证书验证（仅测试环境使用）
# context.check_hostname = False
# context.verify_mode = ssl.CERT_NONE

# 使用自定义上下文创建连接
conn = http.client.HTTPSConnection("external.api.com", context=context)

但本问题核心并非SSL配置问题，优先解决连接复用即可。

总结 #

问题根源是长连接复用失效导致SSL握手异常，程序无法读取到409响应码就抛出错误。通过每次重试新建连接或重置连接状态，即可正常进入409分支处理重试逻辑。

内容的提问来源于stack exchange，提问作者Iraklis Skepasianos