如何在subprocess.Popen子进程崩溃或超时获取stdout和stderr

问题核心 #

使用subprocess.Popen启动子进程后，遇到超时或子进程崩溃时，无法获取完整的stdout和stderr：

• 捕获TimeoutExpired异常时，e.output和e.stderr常为None
• 子进程崩溃时，直接从异常对象无法拿到输出
• proc.kill()无法彻底终止目标进程，只能通过名称匹配杀进程，存在误杀风险

原代码示例 #

主进程代码 #

proc = subprocess.Popen(cmd_full, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True, shell=True, env=env)
try:
    stdout, stderr = proc.communicate(timeout=TIMEOUT_SEC)
    return_code = proc.returncode
    stdout_str = stdout
    stderr_str = stderr
except subprocess.TimeoutExpired as e:
    proc.kill()
    #kill_proc()  # 因为proc.kill()无效，用这个替代
    stdout_str = e.output
    stderr_str = e.stderr
except subprocess.CalledProcessError as e:
    stdout = e.output
    stderr = e.stderr
except Exception as e:
    stdout = e.output
    stderr = e.stderr

自定义终止进程函数 #

def kill_proc():
    process_name = "my_external_proccess.exe"            # 进程名称
    for process in psutil.process_iter(attrs=['pid', 'name']):
        try:
            if process.info['name'].lower() == process_name.lower():
                print(f"{process_name} is already running. Terminating it...")
                process.terminate()
        except (psutil.NoSuchProcess, psutil.AccessDenied, psutil.ZombieProcess):
            pass

解决方案 #

1. 超时场景：手动读取管道剩余内容 #

TimeoutExpired异常的e.output仅包含communicate超时前已读取的部分，需要手动从proc的管道中读取剩余内容：

except subprocess.TimeoutExpired as e:
    # 彻底终止进程树（替代原kill_proc）
    kill_proc_by_pid(proc.pid)
    # 读取剩余的stdout和stderr
    stdout_str = e.output or ""
    stderr_str = e.stderr or ""
    # 补充读取管道中剩余内容
    try:
        stdout_str += proc.stdout.read()
        stderr_str += proc.stderr.read()
    except:
        pass
    # 关闭管道
    proc.stdout.close()
    proc.stderr.close()

2. 崩溃场景：确保拿到完整输出 #

subprocess.CalledProcessError仅在使用check_output等封装函数时才会抛出，直接用Popen时，子进程崩溃后proc.communicate()仍能获取到崩溃前的输出，只需在try块后检查返回码：

try:
    stdout_str, stderr_str = proc.communicate(timeout=TIMEOUT_SEC)
    return_code = proc.returncode
    # 子进程崩溃（非0返回码）时，直接使用已获取的输出
    if return_code != 0:
        # 处理崩溃逻辑，比如记录日志等
        pass
except subprocess.TimeoutExpired as e:
    # 超时处理逻辑同上
    ...
except OSError as e:
    # Popen本身执行失败（进程未启动），此时确实无输出
    stdout_str = ""
    stderr_str = str(e)

3. 修复进程终止逻辑：按PID精准终止进程树 #

原kill_proc按名称杀进程易误杀，改为通过父PID终止整个进程树：

def kill_proc_by_pid(parent_pid):
    try:
        parent = psutil.Process(parent_pid)
        # 获取所有子进程（包括孙子进程）
        children = parent.children(recursive=True)
        # 先终止子进程
        for child in children:
            child.terminate()
        # 再终止父进程
        parent.terminate()
        # 等待进程终止，超时则强制杀死
        gone, alive = psutil.wait_procs(children + [parent], timeout=5)
        for p in alive:
            p.kill()
        print(f"进程树（PID: {parent_pid}）已彻底终止")
    except psutil.NoSuchProcess:
        print("目标进程已不存在")

完整修改后代码 #

import subprocess
import psutil

TIMEOUT_SEC = 30

def kill_proc_by_pid(parent_pid):
    try:
        parent = psutil.Process(parent_pid)
        children = parent.children(recursive=True)
        for child in children:
            child.terminate()
        parent.terminate()
        gone, alive = psutil.wait_procs(children + [parent], timeout=5)
        for p in alive:
            p.kill()
        print(f"进程树（PID: {parent_pid}）已彻底终止")
    except psutil.NoSuchProcess:
        print("目标进程已不存在")

def run_subprocess(cmd_full, cwd, env):
    proc = subprocess.Popen(
        cmd_full,
        cwd=cwd,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        universal_newlines=True,
        shell=True,
        env=env
    )
    stdout_str = ""
    stderr_str = ""
    return_code = -1
    try:
        stdout_str, stderr_str = proc.communicate(timeout=TIMEOUT_SEC)
        return_code = proc.returncode
    except subprocess.TimeoutExpired as e:
        kill_proc_by_pid(proc.pid)
        # 合并已读取和剩余的输出
        stdout_str = (e.output or "") + (proc.stdout.read() if proc.stdout else "")
        stderr_str = (e.stderr or "") + (proc.stderr.read() if proc.stderr else "")
        # 关闭管道
        if proc.stdout:
            proc.stdout.close()
        if proc.stderr:
            proc.stderr.close()
        return_code = -2  # 标记为超时
    except OSError as e:
        stderr_str = f"进程启动失败: {str(e)}"
        return_code = -3  # 标记为启动失败
    finally:
        # 确保管道资源释放
        try:
            if proc.stdout:
                proc.stdout.close()
            if proc.stderr:
                proc.stderr.close()
        except:
            pass
    return return_code, stdout_str, stderr_str

内容的提问来源于stack exchange，提问作者Koby.G