Python Socket多线程服务在Windows下的内存泄漏问题排查
Windows下Python多线程内存泄漏问题
该问题仅在Windows系统Python 3.11.2版本中可复现,macOS下无此现象,暂未测试Linux环境。怀疑Windows平台的threading模块存在异常?由于内存占用持续增长,我添加了del语句与手动垃圾回收操作,移除后内存增长更快。已执行线程join、手动回收,线程也已正常返回,尝试过sys.exit()也无改善。这是一个用于多客户端数据采集的长期运行脚本,以下是可复现该内存问题的完整代码:
threading_mem_leak.py
import threading import socket import gc def main(): ServerSideSocket = socket.socket() host = '' port = 8083 try: ServerSideSocket.bind((host, port)) except socket.error as e: print(str(e)) #print('Socket is listening') ServerSideSocket.listen(5) while True: Client, address = ServerSideSocket.accept() thread_unique_name = address[0] + ":" + str(address[1]) thread = threading.Thread(group=None,target=multi_threaded_client, name=thread_unique_name, args=(Client,), kwargs={}, daemon=None) #print("Connected to: " + address[0] + " : " + str(address[1])) thread.start() print("The thread for connection " + thread.getName() + " has started") #print(threading.enumerate()) thread.join() gc.collect() def multi_threaded_client(connection): data = connection.recv(2048) data.decode('utf-8') if not data: connection.close() del data del connection gc.collect() return connection.close() data1 = str(data) msglist = data1.split('|') var1 = msglist[0][2:] var2 = var1[:-3] var3 = msglist[1] # code that does stuff here # del connection del msglist del data del data1 del var1 del var2 del var3 gc.collect() return if __name__ == "__main__": main()
client.py
import socket while True: ClientSideSocket = socket.socket() try: s = socket.socket() except socket.error as err: print("Socket Error") bytes2send = str.encode("var1|var2|var3") try: s.connect(('127.0.0.1', 8083)) except socket.error as err: print("couldn't connect") try: s.sendall(bytes2send) except: print("couldn't send data") s.close()
已在macOS、Linux运行该脚本(未复现内存问题),尝试过不同位置、不同代的垃圾回收,手动删除对象,也使用过_thread和threading模块,但内存仍持续失控增长。
内容的提问来源于stack exchange,提问作者Unstuckify
相关产品推荐
相关产品推荐

