多进程传递类实例后类属性重置为默认值的问题
问题描述
我创建了一个用于全局存储项目数据的Data类,通过类方法设置root_path,但在多进程的Runner子进程中访问时,root_path始终为空字符串。
Data类代码
class Data: """This class stores project-related constants as class attributes for easy access throughout the project.""" ORG_URL: str = "https://www.myurl.com/" PROJECT_NAME: str = "My Project" root_path: str = "" @classmethod def set_root_path(cls, new_path): cls.root_path = new_path
Main类代码
from multiprocessing import Process, Manager import sys, os class Main: def __init__(self) -> None: # determine if application is a script file or frozen exe if getattr(sys, 'frozen', False): application_path = os.path.dirname(sys.executable) elif __file__: application_path = os.path.dirname(__file__) data = Data() data.set_root_path(f"{application_path}") with Manager() as manager: q = manager.Queue() args = manager.Queue() print(data.root_path) ## prints correct path gui = Process(target=GUI,args=(q,args,data)) runner = Process(target=Runner,args=(q,args,data)) gui.start() runner.start() gui.join() runner.join() if __name__ == "__main__": Main()
Runner类代码
class Runner: def __init__(self,q,args,data) -> None: print("Initiating Runner") self.q = q self.args = args self.data = data print(self.data.root_path) # prints ''
问题原因
Python多进程采用复制内存空间的方式创建子进程,复制操作在子进程启动时完成。主进程中修改Data类的root_path类属性后才启动子进程,但子进程内的Data类是复制的原始版本(root_path为空),类属性的修改无法跨进程同步。同时,传递给子进程的data实例也是一个独立副本,和主进程的实例没有关联。
解决方案
提供三种可行的解决方式:
方法1:使用Manager共享命名空间
利用multiprocessing.Manager创建可跨进程共享的命名空间,存储需要共享的属性:
from multiprocessing import Process, Manager import sys, os class Data: """This class stores project-related constants as class attributes for easy access throughout the project.""" ORG_URL: str = "https://www.myurl.com/" PROJECT_NAME: str = "My Project" @classmethod def init_shared_data(cls, shared_data): shared_data.ORG_URL = cls.ORG_URL shared_data.PROJECT_NAME = cls.PROJECT_NAME class Main: def __init__(self) -> None: # determine if application is a script file or frozen exe if getattr(sys, 'frozen', False): application_path = os.path.dirname(sys.executable) elif __file__: application_path = os.path.dirname(__file__) with Manager() as manager: # 创建共享命名空间 shared_data = manager.Namespace() Data.init_shared_data(shared_data) shared_data.root_path = application_path q = manager.Queue() args = manager.Queue() print(shared_data.root_path) gui = Process(target=GUI, args=(q, args, shared_data)) runner = Process(target=Runner, args=(q, args, shared_data)) gui.start() runner.start() gui.join() runner.join() class Runner: def __init__(self,q,args,shared_data) -> None: print("Initiating Runner") self.q = q self.args = args self.shared_data = shared_data print(self.shared_data.root_path) # 输出正确路径
方法2:直接传递目标参数
如果仅需共享root_path,可以直接将路径作为参数传入子进程,无需传递整个Data实例:
# Main类中修改子进程启动代码 gui = Process(target=GUI, args=(q, args, application_path)) runner = Process(target=Runner, args=(q, args, application_path)) # Runner类修改 class Runner: def __init__(self,q,args,root_path) -> None: print("Initiating Runner") self.q = q self.args = args self.root_path = root_path print(self.root_path) # 输出正确路径
方法3:通过环境变量传递
将root_path设置为主进程的环境变量,子进程会自动继承主进程的环境变量:
# Main类中添加环境变量设置 import os os.environ["PROJECT_ROOT_PATH"] = application_path # Runner类中读取环境变量 class Runner: def __init__(self,q,args,data) -> None: import os print("Initiating Runner") self.q = q self.args = args self.root_path = os.environ.get("PROJECT_ROOT_PATH", "") print(self.root_path) # 输出正确路径
内容的提问来源于stack exchange,提问作者Tester
相关产品推荐
相关产品推荐

