使用multiprocessing无法修改嵌套对象中的数据
问题描述
我正在开发一个个人项目,包含嵌套对象结构:比如World包含多个Environment,每个Environment又包含多个People。我需要把People类中动态变化的属性,逐层传递到Environment类、World类,最终在主程序中展示结果。目前用multiprocessing来运行每个People对象,实现属性的动态修改。
下面是简化后的问题代码:Type1实例化Type2,Type2会修改自身值,Type1希望根据Type2的变化更新自身值,但Type1无法感知到Type2的修改。我试过multiprocessing模块里的locks、manager、values、queues以及syncmanagers,都没解决问题。想问是不是有忽略的关键点?或者有没有更合适的程序结构来实现需求?
我的预期是:Type1根据自身和Type2的change()方法,把特定数据类型的值更新为Type2对应的最新值。
原始代码示例
import multiprocessing import time class Type1: def __init__(self) -> None: self.array =[] self.dict = {} self.text = "" self.number = 0 self.process_dict = {} self.type2_dict = {} self.num = 0 def start(self): new_type = Type2() p = multiprocessing.Process(target=new_type.change) self.process_dict[f"type2{self.num}"] = p self.type2_dict[f"type2{self.num}"] = new_type self.num += 1 p.start() def stop(self): while len(self.process_dict) > 0: _, process = self.process_dict.popitem() process.terminate() process.join() def change(self): self.array, self.dict, self.text, self.number = self.type2_dict[f"type2{self.num-1}"].get_data() def print(self): print(self.array) print(self.dict) print(self.text) print(self.number) print(self.process_dict) class Type2: def __init__(self) -> None: self.array =[] self.dict = {} self.text = "Hello" self.number = 0 def change(self): while True: self.array = [6,7,8,9,10] self.dict = {"d":4,"e":5,"f":6} self.text = "Goodbye" self.number += 1 print("Type2 changed") def get_data(self): return self.array, self.dict, self.text, self.number if __name__ == "__main__": t = Type1() t.start() time.sleep(2) print("\n\nType1 After Start:") t.change() t.print() t.stop() print("\n\nType1 After Finish:") t.print() print("Type1 stopped")
使用Manager后的代码示例(存在问题)
根据建议尝试使用manager后,修改dict时出现None类型,说明不同进程间的shared_var未正确共享。
import multiprocessing import time class Type1: def __init__(self, array, dict, text, number) -> None: self.array = array self.dict = dict self.text = text self.number = number self.process_dict = {} self.type2_dict = {} self.num = 0 def start(self): new_type = Type2(self.array, self.dict, self.text, self.number) p = multiprocessing.Process(target=new_type.change, args=(self.dict,)) self.process_dict[f"type2{self.num}"] = p self.type2_dict[f"type2{self.num}"] = new_type self.num += 1 p.start() def stop(self): while len(self.process_dict) > 0: _, process = self.process_dict.popitem() process.terminate() process.join() def change(self): self.array, self.dict, self.text, self.number = self.type2_dict[f"type2{self.num-1}"].get_data() def print(self): print(self.array.value) print(self.dict.value) print(self.text.value) print(self.number.value) print(self.process_dict) class Type2: def __init__(self, array, dict, text, number) -> None: self.array = array self.dict = dict self.text = text self.number = number def change(self, shared_var): while True: self.array = [6,7,8,9,10] self.text = "Goodbye" self.number = 1 self.dict = shared_var # increase each number in the dictionary by 1 for key in self.dict.keys(): self.dict[key] += 1 # shared_var = self.dict # print("Type2 changed") time.sleep(0.01) def get_data(self): return self.array, self.dict, self.text, self.number if __name__ == "__main__": manager = multiprocessing.Manager() array = manager.list([1,2,3,4,5]) dict = manager.dict({"a":1,"b":2,"c":3}) text = manager.Value("s", "Hello") number = manager.Value("i", 0) t = Type1(array, dict, text, number) t.start() print("\n\nType1 After Start:") t.change() t.print() time.sleep(2) t.stop() print("\n\nType1 After Finish:") t.print() print("Type1 stopped")
内容的提问来源于stack exchange,提问作者Icen
相关产品推荐
相关产品推荐

