subprocess.run调用的CLI输出提前，如何修正输出顺序？

我正在编写一个运行在CI/CD流水线中的Python脚本，流程分为三步：

• 完成配置操作（获取凭证、从服务器拉取文件）
• 使用subprocess.run调用外部CLI工具分析文件
• 将结果发布至其他系统

但遇到日志顺序混乱的问题：CLI工具的输出会提前出现在部分或全部前置日志之前。

简化代码示例 #

import os
import subprocess

print("Fetching CLI configuration")
exit_code, config = fetch_cli_configuration_without_ssl_verification(server_credentials)
if exit_code != 0 or not config:
    print("Error; something went wrong")
    exit(exit_code)
print("Got CLI configuration")

print("Loading result server credentials")
res_server = os.environ["RES_SERVER"]
res_user = os.environ["RES_USER"]
res_pwd = os.environ["RES_PWD"]
if not (res_server and res_user and res_pwd):
    print("Could not load result server credentials")
    exit(1)
print("Loaded credentials for result server", res_server)

print("Running CLI-Tool")
# 该CLI工具会输出内容："I am the CLI"
exit_code = subprocess.run(["mycli", "do", "this", "and", "that"], cwd="somesubdir").returncode
if exit_code != 0:
   print("Error: CLI tool finished with non-zero exit code")
   exit(exit_code)
print("CLI tool finished successfully")

print("Uploading result data")
upload_result_data(read_file_text("cli_results.json"), res_server, res_user, res_pwd)
print("Done uploading result data")

实际异常输出 #

/opt/python-3.10.4/lib/python3.10/site-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'result_server.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  warnings.warn(
I am the CLI
Fetching CLI configuration
Got CLI configuration
Loading result server credentials
Loaded credentials for result server result_server.com
Running CLI-Tool
CLI tool finished successfully
Uploading result data
Done uploading result data

需要确保CLI工具的输出严格出现在Running CLI-Tool日志之后。

问题根源是Python的print输出与子进程输出使用了不同的缓冲策略，导致日志写入终端的顺序错乱。以下是几种可行的解决方法：

方法1：禁用Python全局输出缓冲 #

在脚本开头添加代码强制关闭标准输出缓冲，或者在运行脚本时添加-u参数：

• 脚本内设置：

import sys
import os

# 禁用Python输出缓冲
os.environ['PYTHONUNBUFFERED'] = '1'
# 或者实时刷新标准输出
sys.stdout.flush()

• 运行脚本时指定：

python -u your_script.py

方法2：控制子进程输出的缓冲行为 #

调用subprocess.run时，显式将子进程的输出绑定到父进程的标准流，并设置无缓冲：

import sys

print("Running CLI-Tool")
sys.stdout.flush()  # 先刷新之前的所有print输出

result = subprocess.run(
    ["mycli", "do", "this", "and", "that"],
    cwd="somesubdir",
    stdout=sys.stdout,
    stderr=sys.stderr,
    bufsize=0  # 强制子进程输出无缓冲
)

exit_code = result.returncode
if exit_code != 0:
   print("Error: CLI tool finished with non-zero exit code")
   exit(exit_code)
print("CLI tool finished successfully")

方法3：手动刷新每个print的输出 #

在每个print语句后立即调用sys.stdout.flush()，确保日志实时写入终端：

import sys

print("Fetching CLI configuration")
sys.stdout.flush()

# ... 其他步骤的print后都添加flush
print("Running CLI-Tool")
sys.stdout.flush()

# 再执行subprocess调用

额外处理：urllib3警告提前输出 #

对于urllib3的SSL警告，可以直接禁用避免干扰日志顺序：

from urllib3.exceptions import InsecureRequestWarning
import warnings

warnings.filterwarnings("ignore", category=InsecureRequestWarning)

内容的提问来源于stack exchange，提问作者NoBullsh1t