为何直接以Flask.run为目标调用multiprocessing.Process.start()会报错？

问题场景 #

我正尝试使用flask_restful实现本地SocketServer的加密通信，由于客户端证书管理耗时较长，计划在两个进程、两个端口分别运行一个Flask实例，一个处理HTTP请求，另一个处理HTTPS请求。但直接将app.run作为Process的target时出现报错。

错误代码示例 #

if __name__ == "__main__":
    Process(target=app.run,
            kwargs=dict(host="0.0.0.0",
                        port=5002,
                        ssl_context=('./cert/cert.pem', './cert/cert-key.pem'),
                        debug=False)
            ).start()
    app.run(host="0.0.0.0", port=5001)

报错信息 #

Traceback (most recent call last):
  File "projectDir/main.py", line 80, in <module>
    Process(target=app.run,
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'Flask.__init__.<locals>.<lambda>'

可行解决方案 #

将app.run()封装到一个顶层函数中，再将该函数作为target传入Process：

def run_https(**kwargs):
    app.run(**kwargs)

if __name__ == "__main__":
    Process(target=run_https,
            kwargs=dict(host = "0.0.0.0",
                        port=5002,
                        ssl_context=('./cert/cert.pem', './cert/cert-key.pem'),
                        debug=False)
            ).start()
    app.run(host="0.0.0.0", port=5001)

原因分析 #

Python的multiprocessing启动子进程时，需要通过**序列化（pickle）**将目标函数、参数等传递给子进程：

• 直接传入app.run时，这是一个绑定了app实例的方法。而app内部包含Flask初始化过程中生成的局部lambda函数（比如内部回调或路由相关的匿名函数），这类局部对象无法被pickle序列化，因此触发AttributeError。
• 封装成顶层函数run_https后，序列化的目标变为这个全局可访问的函数。子进程启动后，会在自身进程空间中重新引用app实例并调用run方法，避开了对绑定方法及内部局部对象的序列化要求，因此可以正常运行。

内容的提问来源于stack exchange，提问作者EddiKh