Python多进程Ctrl+C退出后进程残留及资源占用问题

我的Python应用同时使用了线程执行器和带线程的多进程机制，按下Ctrl+C尝试回到命令行提示符时，多进程队列触发pickle报错，导致提示符无法恢复。

报错信息 #

Traceback (most recent call last):
  File "/home/juanmf/projects/turret/env/lib/python3.11/site-packages/multiprocess/queues.py", line 248, in _feed
    obj = _ForkingPickler.dumps(obj)
...
  File "/home/juanmf/projects/turret/env/lib/python3.11/site-packages/dill/_dill.py", line 414, in save
    StockPickler.save(self, obj, save_persistent_id)
  File "/usr/lib/python3.11/pickle.py", line 578, in save
    rv = reduce(self.proto)
         ^^^^^^^^^^^^^^^^^^
TypeError: cannot pickle '_queue.SimpleQueue' object

当前处理方式及残留问题 #

暂时不想修复pickle问题，改用SIGKILL信号强制终止进程，虽然能回到命令行，但仍有Python进程残留并占用树莓派摄像头资源，导致应用无法重启。

执行SIGKILL后的进程列表：

Thread prints Dump Complete =======================================================

Killed
(env) juanmf@raspberrypi:~/projects/turret/src $ ps aux | grep pyth
root         956  0.0  0.4 102112 19256 ?        Ss   19:52   0:00 python /usr/sbin/wayvnc-control.py
juanmf      1100  0.0  0.8  63164 34052 ?        S    19:52   0:00 /usr/bin/python3 /usr/share/system-config-printer/applet.py
juanmf      2284  0.0  2.1 1185628 83564 pts/0   Sl   20:18   0:00 python3 Training.py
juanmf      2291  0.0  2.1 1251204 83280 pts/0   Sl   20:18   0:00 python3 Training.py
juanmf      2298  0.0  2.1 1316784 83304 pts/0   Sl   20:18   0:00 python3 Training.py

注：应用仅包含主进程+1个子进程，但杀死主进程后仍残留3个进程，必须执行pgrep python3 | xargs kill -9才能重启应用。

强制终止代码 #

# 注册中断信号处理函数（Ctrl+C或kill）
signal.signal(signal.SIGINT, interrupt_handler)
signal.signal(signal.SIGTERM,  interrupt_handler)

...
def interrupt_handler(signum, frame):
    try:
        for process, proxyController in MultiProcessingControllerFactory.runningProcesses:
            print(process.pid, signal.SIGKILL)
            process.terminate()
            os.kill(process.pid, signal.SIGKILL)

        process = current_process()
        # 退出当前进程
        if process._popen is not None:
            # 访问受保护属性，因为multiprocessing未做安全检查
            process.terminate()

        GPIO.cleanup()
        flush_streams()  # 打印缓冲日志
        time.sleep(2)
    finally:
        os.kill(os.getpid(), signal.SIGKILL)  # 杀死当前进程
        exit(0)

补充进程信息 #

应用支持单进程/多进程模式，多进程模式下仅启动1个子进程（Process.start()的执行逻辑仅打印一次"SPAWNING!!"）。按下Ctrl+Z暂停应用后，进程列表显示：

单进程模式输出

[1]+  Stopped                 python3 Training.py
(env) juanmf@raspberrypi:~/projects/turret/src $ ps aux | grep pyth
root         956  0.0  0.4 102112 19256 ?        Ss   19:52   0:00 python /usr/sbin/wayvnc-control.py
juanmf      1100  0.0  0.8  63164 34052 ?        S    19:52   0:00 /usr/bin/python3 /usr/share/system-config-printer/applet.py
juanmf      2078 43.0  3.6 1246880 142360 pts/0  TLl  20:12   0:06 python3 Training.py

多进程模式输出

[1]+  Stopped                 python3 Training.py
(env) juanmf@raspberrypi:~/projects/turret/src $ ps aux | grep pyth
root         956  0.0  0.4 102112 19256 ?        Ss   19:52   0:00 python /usr/sbin/wayvnc-control.py
juanmf      1100  0.0  0.8  63164 34052 ?        S    19:52   0:00 /usr/bin/python3 /usr/share/system-config-printer/applet.py
juanmf      2269 43.3  3.7 1468804 144136 pts/0  TLl  20:18   0:07 python3 Training.py
juanmf      2284  0.1  2.1 1185644 83564 pts/0   Tl   20:18   0:00 python3 Training.py
juanmf      2291  0.0  2.1 1251204 83280 pts/0   Tl   20:18   0:00 python3 Training.py
juanmf      2298  0.0  2.1 1316784 83304 pts/0   Tl   20:18   0:00 python3 Training.py
juanmf      2306  0.0  2.1 1464968 83636 pts/0   Tl   20:18   0:00 python3 Training.py

核心疑问 #

强制终止后残留3个进程，推测主进程和子进程已终止，但这3个未注册的进程来自何处？（另外，消失的是第一个和最后一个PID）

内容的提问来源于stack exchange，提问作者juanmf