Python subprocess.run触发UnicodeDecodeError问题求助

问题概述 #

FastAPI应用中包含4个调用PowerShell脚本操作Active Directory（AD）的API，第一个API的错误处理逻辑正常运行，但复制相同结构到第二个API后，触发UnicodeDecodeError错误，报错信息如下：

File "C:\dfb-apis\main.py", line 361, in manageadgroup
result = subprocess.run(["powershell.exe",
File "C:\Program Files\Python310\lib\subprocess.py", line 505, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "C:\Program Files\Python310\lib\subprocess.py", line 1144, in communicate
stderr = self.stderr.read()
File "C:\Program Files\Python310\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 55: character maps to <undefined>

调用流程为：Python应用 → ADMgmtInitial.ps1 → 特权用户上下文执行第二个AD操作脚本，以此获取特定OU的操作权限。

错误原因 #

1. 编码不匹配：Windows系统默认的Python subprocess编码为cp1252，但PowerShell输出的内容包含该编码无法解码的字节（如0x81）——第二个API对应的AD操作可能涉及带特殊字符的组名/用户名，或错误信息包含非cp1252兼容符号，而第一个API的输出恰好无此类字符，因此未触发错误。
2. PowerShell输出捕获逻辑缺陷：ADMgmtInitial.ps1中使用Start-Process调用第二个脚本，默认无法捕获子进程的输出，导致错误信息传递异常，进一步引发编码解析问题。
3. 输出流混淆：Python代码仅捕获stderr，但PowerShell的Write-Host输出直接发送到控制台而非标准流，导致实际错误信息未被正确捕获，反而产生乱码字节。

修复步骤 #

1. 统一Python subprocess的编码设置 #

修改subprocess.run调用，明确指定UTF-8编码，并添加错误处理避免崩溃：

result = subprocess.run(
    ["powershell.exe", 
     "./managead/ADMgmtInitial.ps1", 
     "-action", item.action, 
     "-groupname", providedadgroup, 
     "-techuser", item.techuser,
     "-region", item.region],  
    text=True,
    stdout=subprocess.PIPE,  # 同时捕获标准输出
    stderr=subprocess.PIPE,
    encoding='utf-8',  # 强制使用UTF-8解码
    errors='replace'   # 无法解码的字符用�替换，避免崩溃
)

2. 修复PowerShell脚本的输出捕获逻辑 #

替换Start-Process为Invoke-Command，直接在特权上下文执行脚本并捕获输出：

try {
    # 传递参数并在特权用户上下文执行AD操作脚本
    $output = Invoke-Command -Credential $credential -ScriptBlock {
        param($scriptPath, $action, $groupname, $techuser, $region)
        & $scriptPath -action $action -groupname $groupname -techuser $techuser -region $region
    } -ArgumentList $ArgumentList[0], $ArgumentList[1], $ArgumentList[3], $ArgumentList[5], $ArgumentList[7]
    
    Write-Output "PS1 script ADMgmtInitial: Execution output: $output"
    $exitCode = 0
} catch {
    $errorMessage = $_.Exception.Message
    Write-Output "PS1 script ADMgmtInitial: Error: $errorMessage"
    Write-Output "PS1 script ADMgmtInitial: The AD Management script has failed"
    throw $errorMessage
}

若必须使用Start-Process，需添加输出重定向，将子进程的stdout/stderr写入临时文件后读取：

$tempOut = New-TemporaryFile
$tempErr = New-TemporaryFile
try {
    $process = Start-Process powershell.exe -Credential $credential -ArgumentList $ArgumentList `
        -NoNewWindow -Wait -RedirectStandardOutput $tempOut.FullName -RedirectStandardError $tempErr.FullName
    $exitCode = $process.ExitCode
    # 读取输出并转为UTF-8
    $output = Get-Content $tempOut.FullName -Raw -Encoding UTF8
    $errOutput = Get-Content $tempErr.FullName -Raw -Encoding UTF8
    Write-Output "PS1 script ADMgmtInitial: exitCode: $exitCode"
    if ($output) { Write-Output $output }
    if ($errOutput) { Write-Output $errOutput }
} catch {
    $errorMessage = $_.Exception.Message
    Write-Output "PS1 script ADMgmtInitial: Error: $errorMessage"
    Write-Output "PS1 script ADMgmtInitial: The AD Management script has failed"
    throw $errorMessage
} finally {
    Remove-Item $tempOut, $tempErr -Force -ErrorAction SilentlyContinue
}

3. 替换PowerShell的Write-Host为Write-Output #

Write-Host直接输出到控制台，无法被subprocess捕获，改为Write-Output将内容发送到标准输出流：

# 第二个AD操作脚本修改示例
Try {
    Add-ADGroupMember -Identity $groupname -Members $techuser;
} catch {
    $errorMessage = $_.Exception.Message
    Write-Output "PS1 script ADgroupmgmt: Error: $errorMessage"
    Write-Output "PS1 script ADgroupmgmt: The AD Management script has failed"
    $ErrorActionPreference = 'Stop'
    throw $errorMessage
}  

4. 优化Python的结果处理逻辑 #

同时读取stdout和stderr，避免遗漏错误信息：

if result.returncode == 0:
    returncodestring = str(result.returncode)
    print(f"returncode in Python: {returncodestring}")
    
    # 合并stdout和stderr的信息
    message = f"{result.stdout.strip()}\n{result.stderr.strip()}"
    print(f"Execution message:\n{message}")

    if item.action == "add":
        success_msg = f"Success: User {item.techuser} has been added to the AD group {providedadgroup}"
    else:
        success_msg = f"Success: User {item.techuser} has been removed from the AD group {providedadgroup}"
    print(success_msg)
    return {"message": success_msg}
else:
    returncodestring = str(result.returncode)
    print(f"returncode in Python: {returncodestring}")
    # 合并错误信息
    error_msg = f"{result.stderr.strip()}\n{result.stdout.strip()}"
    print(f"Python Error | Error message: {error_msg}")
    raise HTTPException(
        status_code=500,
        detail=f"Failed to {item.action} user {item.techuser} to/from AD group {providedadgroup}. Details: {error_msg}",
        headers={"Error": f"Could not {item.action} user {item.techuser} to/from AD group {providedadgroup}"},
    )

额外优化建议 #

• 在所有PowerShell脚本开头添加编码强制设置，确保输出为UTF-8：

  $OutputEncoding = [Console]::OutputEncoding = [System.Text.UTF8Encoding]::new()
  

• 统一所有API的subprocess调用参数，避免因编码不一致导致的偶发问题；
• 使用f-string拼接Python字符串，替代加号拼接，提升可读性并减少编码风险。

内容的提问来源于stack exchange，提问作者Eleandro