Windows环境下Python TCP连接未清理致服务器无响应问题排查

问题背景 #

• 在AWS Windows服务器上运行Python IoT数据服务，运行一段时间后卡在s.accept()调用，无法处理新请求，排查发现TCP连接过多，操作系统无法分配新连接。
• 日志显示所有已accept的连接都对应有关闭记录，且活跃线程数为0，但Windows资源监视器中Python进程仍存在50+未关闭的TCP连接。

更新排查结果 #

• 未关闭的连接从未被服务器accept过，确来自IoT设备；通过netstat -an确认这些连接均处于CLOSE_WAIT状态。
• 需求：强制清理Windows上处于CLOSE_WAIT状态超过5分钟的连接。

相关代码示例 #

def connection(conn, addr):
    conn.settimeout(10)
    data = None
    connection_time = datetime.now()
    n_items = 0
    try:
        print(connection_time.strftime("[%d/%m/%Y, %H:%M:%S] "), "new connection started:", addr)
        data = get_info(conn)
        print(addr, data)
        
        # serve client here, protocol omitted

    except Exception as e:
        print(f"{addr} connection error:" + str(e))
    if data is not None:
        add_connection_info(addr, data, connection_time)
    try:
        conn.close()
        print(connection_time.strftime("[%d/%m/%Y, %H:%M:%S] "), "connection ended:", addr)
    except Exception as e:
        print(f"close failed: {addr} ; {e}")


if __name__ == '__main__':
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
    ssl_context.load_cert_chain(cert, key, password=*omitted*)
    s = socket.socket()
    s = ssl_context.wrap_socket(s, server_side=True)
    host = "0.0.0.0" 
    port = 12345  # not the actual port

    print('Server started:', host, port)

    s.bind((host, port))  # Bind to the port
    s.listen()  # Now wait for client connection.
    s.setblocking(False)
    # Join completed threads and check connection status
    threads = []
    while True:
        for thread in threads:
            thread.join(0)
        threads = [t for t in threads if t.is_alive()]
        print(f"{len(threads)} active connections")
        try:
            # Use select to wait for a connection or timeout
            rlist, _, _ = select.select([s], [], [], 100)  # 100 seconds timeout
            if s in rlist:
                s.settimeout(10)
                # TODO timout here
                c, addr = s.accept()
                print(f"Accepted connection from {addr}")
                thread = Thread(target=connection, args=(c, addr))
                #thread.daemon = True
                thread.start()
                threads.append(thread)
                print("thread started")
            else:
                print("No connection within 100 second period")

        except BlockingIOError:
            print("No connection ready")
        except Exception as e:
            print("error", str(e))
            
            try:
                c.close()
                print(f"Connection from {addr} closed due to error.")
            except Exception as e_close:
                print(f"Failed to close connection after error: {str(e_close)}")

解决方案 #

一、操作系统层面处理CLOSE_WAIT连接 #

Windows没有原生自动清理超时CLOSE_WAIT连接的命令，但可通过以下方式优化：

1. 调整TCP参数缩短超时周期
   • 打开注册表编辑器（regedit），定位到HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
   • 添加/修改以下DWORD值：
     • TcpTimedWaitDelay：设为十进制300（5分钟，单位秒），控制连接处于TIME_WAIT的超时（辅助加速CLOSE_WAIT回收）
     • MaxConnectionsPerServer：根据服务器配置设置合理上限，避免连接耗尽
   • 修改后重启服务器生效
2. 手动强制关闭（紧急情况）
   • 执行netstat -ano | findstr :<你的端口> | findstr CLOSE_WAIT，找到对应连接的PID
   • 执行taskkill /F /PID <目标PID>杀死对应进程（注意：会终止整个Python服务）

二、代码层面修复未accept连接堆积 #

问题根源是监听socket未处理未完成的三次握手连接，导致系统堆积CLOSE_WAIT连接，修改方向：

1. 设置监听队列上限
   在s.listen()时指定合理backlog值，比如s.listen(100)，控制等待accept的连接队列长度，避免队列溢出。
2. 优化accept超时与异常处理
   为s.accept()的超时添加捕获逻辑，避免未处理的连接残留；同时确保异常路径中不会引用未定义的socket变量。
3. 线程与socket资源优化
   • 设置线程为守护线程（thread.daemon = True），避免线程残留导致资源泄漏
   • 清理冗余的线程join逻辑，简化活跃线程维护

修改后的核心代码片段：

if __name__ == '__main__':
    # ... 其他初始化代码 ...
    s.listen(100)  # 指定监听队列长度
    s.setblocking(False)
    threads = []
    while True:
        # 直接过滤活跃线程，简化逻辑
        threads = [t for t in threads if t.is_alive()]
        print(f"{len(threads)} active connections")
        try:
            rlist, _, _ = select.select([s], [], [], 100)
            if s in rlist:
                s.settimeout(10)
                try:
                    c, addr = s.accept()
                    print(f"Accepted connection from {addr}")
                    thread = Thread(target=connection, args=(c, addr))
                    thread.daemon = True  # 设置守护线程
                    thread.start()
                    threads.append(thread)
                    print("thread started")
                except socket.timeout:
                    print("Accept timed out, skip uncompleted connection")
            else:
                print("No connection within 100 second period")
        except BlockingIOError:
            print("No connection ready")
        except Exception as e:
            print("error", str(e))
            # 仅当socket变量存在时执行关闭
            if 'c' in locals():
                try:
                    c.close()
                    print("Connection closed due to error.")
                except Exception as e_close:
                    print(f"Failed to close connection after error: {str(e_close)}")

三、IoT设备端排查 #

CLOSE_WAIT状态通常由客户端未主动关闭连接导致，需排查设备逻辑：

• 确保设备在请求完成后主动调用连接关闭接口
• 设备端设置合理的连接超时，避免异常场景下连接挂起

内容的提问来源于stack exchange，提问作者drulludanni